Authentication using Delegation Tokens
Authentication based on delegation tokens is a lightweight authentication mechanism that you can use to complement existing
SASL/SSL methods. Delegation tokens are shared secrets between Apache Kafka® brokers and clients. Delegation tokens can help frameworks
to distribute the workload to available workers in a secure environment without the added cost of distributing
Kerberos TGT/keytabs or keystores when 2-way SSL is used.
The typical steps for delegation token usage are:
- You authenticate with the Kafka cluster via SASL or SSL, and obtain a delegation token. You can do this using the
AdminClient APIs or the
kafka-delegation-tokens
script.
- You provide the delegation token to Kafka clients securely for authenticating with the Kafka cluster.
- The token owner or renewer can renew or expire the delegation tokens.
Token Management
A primary secret key is used to generate and verify delegation tokens. This is supplied using config
option delegation.token.master.key
. The same secret key must be configured across all the brokers.
If the secret is not set or is set to an empty string, the brokers will disable the delegation token authentication.
In the current implementation, token details are stored in ZooKeeper and they can be used in Kafka
installations where ZooKeeper is on a private network. The primary secret key is stored as
plain text in the server.properties
config file.
A token has a current life, and a maximum renewable life. By default, tokens must be renewed once every 24 hours
for up to 7 days. You can configure the token life by using the delegation.token.expiry.time.ms
and
delegation.token.max.lifetime.ms
config options.
You can also explicitly cancel tokens. If a token is not renewed by the token’s expiration time or if token
is beyond the maximum life time, it will be deleted from all broker caches and from ZooKeeper.
Creating Delegation Tokens
You can create tokens by using AdminClient APIs or by using kafka-delegation-tokens
script.
Delegation token requests (create/renew/expire/describe) must be issued on SASL or SSL authenticated channels.
Tokens cannot be requested if the initial authentication is done by using a delegation token.
kafka-delegation-tokens
script examples are given below.
Create a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --create --max-life-time-period -1 \
--command-config client.properties --renewer-principal User:user1
Renew a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --renew --renew-time-period -1 \
--command-config client.properties --hmac ABCDEFGHIJK
Expire a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --expire --expiry-time-period -1 \
--command-config client.properties --hmac ABCDEFGHIJK
Existing tokens can be described using the --describe
option:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --describe \
--command-config client.properties --owner-principal User:user1
Token Authentication
Delegation token authentication piggybacks on the current SASL/SCRAM authentication mechanism.
You must enable SASL/SCRAM mechanism on Kafka cluster as described here.
Configuring Kafka Clients
You can configure the JAAS configuration property for each client in producer.properties
or consumer.properties
files. The login module describes how the clients like producer and consumer can
connect to the Kafka broker. The following is an example configuration for a client for the token authentication:
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
username="tokenID123" \
password="lAYYSFmLs4bTjf+lTZ1LCHR/ZZFNA==" \
tokenauth="true";
The options username
and password
are used by clients to configure the token ID and
token HMAC. And the option tokenauth
is used to indicate the server about token authentication.
In this example, clients connect to the broker using token ID: tokenID123
. Different clients within a
JVM may connect using different tokens by specifying different token details in sasl.jaas.config
.
Procedure to Manually Rotate the Secret
You must redeploy your Kafka cluster when the secret is rotated. During this process,
currently connected clients will continue to work, but any new connection requests and
renew and expire requests with old tokens can fail. Here are the steps:
- Expire all existing tokens.
- Rotate the secret by rolling upgrade.
- Generate new tokens.
Notes on Delegation Tokens
Currently, you can only create a delegation token for yourself. Owner and renewers can renew or expire tokens.
Owner and renewers can describe their own tokens. To describe others tokens, you must add DESCRIBE
permission on Token Resource.