Licensing Connectors
With a Developer license, you can use Confluent Platform commercial connectors on an
unlimited basis, in Connect clusters that use a single-broker Apache Kafka®
cluster. For Connect clusters that use a multi-broker Kafka cluster, you can
use commercial connectors for a 30-day trial without an enterprise license key.
After 30 days, Confluent Platform software stops working, as described in commercial
components post-expiry. You must
purchase an enterprise license to continue using
commercial connectors in Connect clusters that use a multi-broker Kafka
cluster.
Refer to the individual supported connector
documentation for connector-specfic information about license requirements.
Centralized License in the Kafka Connect Worker
Prior to the release of Confluent Platform version 6.0, you were required to include the
license-related properties in the
configuration for each Confluent Platform commercial connector you used. However, beginning
with Confluent Platform version 6.0, you can now put these license-related properties in the
Connect worker configuration instead of in each connector configuration.
Note
The Centralized License is a proprietary feature and not available in Community licensed distributions.
When the Connect worker configuration includes the license-related
properties, the worker automatically injects the license-related properties into
each connector configuration that requires a license, but that does not
already include the license-related properties.
Important
The Kafka Connect worker only injects license-related properties into
connector configurations that need them. It does not inject license-related
properties if the connector configuration already includes license-related
properties or if the connector does not require a license.
The license-related properties are injected when the connector is started.
License properties are never persisted outside of the worker configuration. To
change any of the license-related properties passed to the connectors, simply
change the license-related properties in the worker configuration and restart
the worker. The worker will restart any assigned connectors after injecting the
worker’s latest license-related properties.
Note
It is important to set the license-related properties in each Connect
worker configuration. You can change the license-related properties and use a
rolling restart to have those properties take effect across the cluster.
Connect Worker License Properties
Both Connect standalone and distributed configurations support the same
license-related properties that are used
in commercial connector configurations. These license-related properties
include:
confluent.license
: The Confluent enterprise license key string for your organization. You don’t need to add this if you are using a Developer license, a Trial (Evaluation) License, or a Control Center License.
confluent.topic
: The name of the Kafka topic where the license is stored. This defaults to _confluent-command
.
confluent.topic.replication.factor
: This is used only if the license topic specified in confluent.topic
does not exist when the license checks are performed. This specifies the replication factor that is used if the license check creates the topic. This property defaults to 3
.
confluent.topic.bootstrap.servers
: The bootstrap addresses for the Kafka cluster where the license topic is stored. By default, the worker uses the value of bootstrap.servers
, which is the Kafka cluster that Connect uses. Note that this entry is only needed if the license topic is stored on a different Kafka cluster than the Kafka cluster that Connect uses.
confluent.topic.*
: Other producer and consumer properties that are required to read and write to the topic specified in confluent.topic
at the Kafka cluster specified in confluent.topic.bootstrap.servers
. By default, all of the producer and consumer properties are configured without prefixes in the Connect worker configuration. For example, if the broker uses SSL for connections and the worker configuration includes properties such as ssl.keystore.location
and ssl.keystore.password
, then these will be inferred as confluent.topic.ssl.keystore.location
and confluent.topic.ssl.keystore.password
, respectively.
By default the Connect worker uses the same settings and the same Kafka
principal for reading and writing to the _confluent-command
license topic
that it uses to read and write Connect’s internal topics. If the topic does
not exist when it’s needed, the topic is created with a replication factor of
3 (a sensible default for production).
Note
When using Connect in development with a Kafka cluster that has one
broker, include confluent.topic.replication.factor=1
so that if a
license topic is needed it is created with a single replica.
You can always set any of the confluent.topic.*
properties in the worker configuration, even if the value is the same as
what would be inferred by the worker from its other properties.
Adding or Changing Connect Worker License Properties
The default license-properties inferred by the Connect worker may not be
sufficient. You may need to set specific license-related properties in all your
worker configurations. To do this on a running cluster, change the worker
configuration for one worker, restart that worker, and verify all
currently-defined connectors and tasks are still healthy. Repeat this on the
other workers in the same Connect cluster.
Note
It is important to set the license-related properties in each worker
configuration in the Connect cluster. If you don’t, it’s possible that a
connector will run successfully when assigned to one worker, but then fails
when assigned to a different worker.
You can then start a new licensed connector without defining the license-related
properties in the individual connector configurations.
Tip
You can remove the license-related properties from any of your existing
licensed connector configurations. Do this one connector at a time. Make sure
that after you remove the license-related properties from a connector, that
you verify the connector starts with no license-related properties specified
in the connector configuration.
If a connector fails to start because of a license-related problem, check the
worker logs for the connector configuration. The logs will include the
license-related properties injected by the worker. Determine which
license-related properties need to be changed and then apply the changes to
those worker configs as described above.
Security configuration properties
confluent.topic.ssl.truststore.location
The location of the trust store file.
- Type: string
- Default: null
- Importance: high
confluent.topic.ssl.truststore.password
The password for the trust store file. If a password is not set access to the truststore is still available, but
integrity checking is disabled.
- Type: password
- Default: null
- Importance: high
confluent.topic.ssl.keystore.location
The location of the key store file. This is optional for client and can be used for two-way authentication for client.
- Type: string
- Default: null
- Importance: high
confluent.topic.ssl.keystore.password
The store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured.
- Type: password
- Default: null
- Importance: high
confluent.topic.ssl.key.password
The password of the private key in the key store file. This is optional for client.
- Type: password
- Default: null
- Importance: high
confluent.topic.security.protocol
Protocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.
- Type: string
- Default: “PLAINTEXT”
- Importance: medium
License topic configuration
A Confluent enterprise license is stored in the _confluent-command
topic.
This topic is created by default and contains the license that corresponds to
the license key supplied through the confluent.license
property.
Note
No public keys are stored in Kafka topics.
The following describes how the default _confluent-command
topic is
generated under different scenarios:
- A 30-day trial license is automatically generated for the
_confluent command
topic if you do not add the confluent.license
property or leave this property empty (for example, confluent.license=
).
- Adding a valid license key (for example,
confluent.license=<valid-license-key>
) adds a valid license in the _confluent-command
topic.
Here is an example of the minimal properties for development and testing.
You can change the name of the _confluent-command
topic using the
confluent.topic
property (for instance, if your environment has strict
naming conventions). The example below shows this change and the configured
Kafka bootstrap server.
confluent.topic=foo_confluent-command
confluent.topic.bootstrap.servers=localhost:9092
The example above shows the minimally required bootstrap server property that
you can use for development and testing. For a production environment, you add
the normal producer, consumer, and topic configuration properties to the
connector properties, prefixed with confluent.topic.
.
License topic ACLs
The _confluent-command
topic contains the license that corresponds to the
license key supplied through the confluent.license
property. It is created
by default. Connectors that access this topic require the following ACLs
configured:
- CREATE and DESCRIBE on the resource cluster, if the connector needs to create the topic.
- DESCRIBE, READ, and WRITE on the
_confluent-command
topic.
You can provide access either individually for each principal that will
use the license or use a wildcard entry to
allow all clients. The following examples show commands that you can use to
configure ACLs for the resource cluster and _confluent-command
topic.
Set a CREATE and DESCRIBE ACL on the resource cluster:
kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf \
--add --allow-principal User:<principal> \
--operation CREATE --operation DESCRIBE --cluster
Set a DESCRIBE, READ, and WRITE ACL on the _confluent-command
topic:
kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf \
--add --allow-principal User:<principal> \
--operation DESCRIBE --operation READ --operation WRITE --topic _confluent-command