CONFLUENT PLATFORM
In an RBAC-enabled environment, you must create a service principal for the Connect cluster. Within the service principal, you create role bindings so the Connect cluster can access the Kafka cluster and other resources.
The following steps guide you through configuring a service principal for Kafka Connect. How you create the service principal depends on your organization and operating environment (on-premises, private cloud, or public cloud) and is not covered in this procedure.
Note
For instructions on getting your actual cluster IDs, refer to Discover Identifiers for Clusters.
Enter the following Confluent CLI command to give an example service principal named $CONNECT_USER the role SecurityAdmin on the Connect cluster. The example cluster IDs $CONNECT_CLUSTER for Connect and $KAFKA_CLUSTER for Kafka are used in the command and all subsequent command examples.
$CONNECT_USER
SecurityAdmin
$CONNECT_CLUSTER
$KAFKA_CLUSTER
confluent iam rolebinding create \ --principal User:$CONNECT_USER \ --role SecurityAdmin \ --kafka-cluster-id $KAFKA_CLUSTER \ --connect-cluster-id $CONNECT_CLUSTER
Enter the following command to give $CONNECT_USER the role ResourceOwner on the group that Connect workers use to coordinate with other workers.
ResourceOwner
confluent iam rolebinding create \ --principal User:$CONNECT_USER \ --role ResourceOwner \ --resource Group:$CONNECT_CLUSTER \ --kafka-cluster-id $KAFKA_CLUSTER
Enter the following commands to give $CONNECT_USER the role ResourceOwner on the internal Kafka topics used by Connect to store configuration, status, and offset information. Internal configuration topic $CONFIGS_TOPIC, internal offsets topic $OFFSETS_TOPIC, and status topic $STATUS_TOPIC are used in the examples.
$CONFIGS_TOPIC
$OFFSETS_TOPIC
$STATUS_TOPIC
The configuration topics config.storage.topic, offset.storage.topic, and status.storage.topic are where the internal configuration, offset configuration, and status configuration data are stored. These are set for Confluent Platform to connect-configs, connect-offsets, and connect-status by default.
config.storage.topic
offset.storage.topic
status.storage.topic
connect-configs
connect-offsets
connect-status
confluent iam rolebinding create \ --principal User:$CONNECT_USER \ --role ResourceOwner \ --resource Topic:$CONFIGS_TOPIC \ --kafka-cluster-id $KAFKA_CLUSTER
confluent iam rolebinding create \ --principal User:$CONNECT_USER \ --role ResourceOwner \ --resource Topic:$OFFSETS_TOPIC \ --kafka-cluster-id $KAFKA_CLUSTER
confluent iam rolebinding create \ --principal User:$CONNECT_USER \ --role ResourceOwner \ --resource Topic:$STATUS_TOPIC \ --kafka-cluster-id $KAFKA_CLUSTER
Important
By default the Kafka worker uses the same settings and the same principal for reading and writing to the _confluent-command license topic that it uses to read and write to internal topics. For more information about Connect licensing, see Licensing Connectors.
_confluent-command
If you have configured a Connect Secret Registry, you must complete two additional steps.
Enter the following command to give $CONNECT_USER the role ResourceOwner on the group that secret registry nodes use to coordinate with other nodes. Secret Registry group ID $SECRET_REGISTRY_GROUP is used in the example. Note that the actual value of $SECRET_REGISTRY_GROUP needs to match the value of config.providers.secret.param.secret.registry.group.id in the Connect worker properties. This value defaults to secret-registry if not specified in the Connect worker properties.
$SECRET_REGISTRY_GROUP
config.providers.secret.param.secret.registry.group.id
secret-registry
confluent iam rolebinding create \ --principal User:$CONNECT_USER \ --role ResourceOwner \ --resource Group:$SECRET_REGISTRY_GROUP \ --kafka-cluster-id $KAFKA_CLUSTER
Enter the following command to give $CONNECT_USER the role ResourceOwner on the Kafka topic used to store secrets. Kafka secrets topic $SECRETS_TOPIC is used in the example. Note that the actual value of $SECRETS_TOPIC needs to match the value of config.providers.secret.param.kafkastore.topic in the Connect worker properties. This value defaults to _confluent-secrets if not specified in the Connect worker properties.
$SECRETS_TOPIC
config.providers.secret.param.kafkastore.topic
_confluent-secrets
Warning
The default value for the secrets topic changed from _secrets to _confluent-secrets in version 5.4. If your Secret Registry cluster is not configured with a kafkastore.topic property, explicitly set it to _secrets before upgrading to 5.4, to avoid losing existing secrets.
_secrets
kafkastore.topic
confluent iam rolebinding create \ --principal User:$CONNECT_USER \ --role ResourceOwner \ --resource Topic:$SECRETS_TOPIC \ --kafka-cluster-id $KAFKA_CLUSTER