Confluent Platform secrets allow you to store and manage sensitive information, such as passwords and API tokens. Compliance
requirements often dictate that services should not store sensitive data as clear text in files. This sensitive data
can include passwords, such as in the configuration parameters ssl.key.password
, ssl.keystore.password
,
ssl.truststore.password
, or any other sensitive data in configuration files. Secrets are administered with the
confluent secret commands.
Secrets use “envelope encryption”, which is a standard way to protect
sensitive data with a highly secure method. A user specifies
a master passphrase, which is used along with a cryptographic salt value to derive
a master encryption key. The master encryption key is used to generate a separate
data encryption key.
The master encryption and data encryption keys are used to encrypt the sensitive
data in the configuration files. The service can later decrypt the keys. If an
unauthorized user gains access to a configuration file, they cannot see the
encrypted values and they have no way to decrypt them without knowing the master
encryption key.
Production
To operationalize this workflow, you can augment your orchestration tooling to distribute this to
the destination hosts. These hosts may include Kafka brokers, Connect workers, Schema Registry instances, ksqlDB servers, Control Center,
or any service using password encryption. The confluent secret commands are flexible to accommodate whatever
secret distribution model you prefer. You can either do the secret generation and configuration modification on each
destination host directly, or do it all on a single host and then distribute the secret data to the destination hosts. Here are
the tasks to distribute the secret data:
- Export the master encryption key into the environment on every host that will have a configuration file with password
protection.
- Distribute the secrets file: copy the secrets file
/path/to/security.properties
from the local host on which you
have been working to /path/to/security.properties
on the destination hosts.
- Propagate the necessary configuration file changes: update the configuration file on all hosts so that the
configuration parameter now has the tuple for secrets.
Usage examples
Here are some usage examples for the confluent secret commands.
Add remote encrypted configs from file
This command adds new encrypted configuration parameters (--config
) to the
specified file (--config-file
). The encrypted secrets are stored in the local secrets file (--local-secrets-file
).
Tip
You can specify multiple key-value pairs by separating each configuration parameter with a newline character,
for example: --config "ssl.keystore.password = sslPassword \n ssl.truststore.password = password"
.
If you include config group.id
but do not include a key/value pair, or submit an empty list, you will receive an output error.
confluent secret file add --config-file /etc/kafka/connect-distributed.properties \
--local-secrets-file /usr/secrets/security.properties \
--remote-secrets-file /usr/secrets/security.properties \
--config group.id=connect-cluster
After running this command your properties file should resemble:
group.id = ${securepass:/usr/secrets/security.properties:connect-distributed.properties/group.id}
Using prefixes in secrets configurations
Secrets config.providers
do not propagate to prefixes such as client.*
.
Thus, when using prefixes with secrets you must specify config.providers
and config.providers.securepass.class
:
client.config.providers=securepass
client.config.providers.securepass.class=io.confluent.kafka.security.config.provider.SecurePassConfigProvider
Encrypt JAAS configuration parameters
All Confluent Platform components support an embedded JAAS configuration, which provides a secure
way to specify your JAAS configuration. Secrets also support using an embedded JAAS
configuration. This configuration method allows for a more granular level of
security for your secrets JAAS configuration. For example, rather than encrypting
your entire secrets JAAS configuration, as you would using a properties file,
when using an embedded JAAS configuration, you can encrypt the password only. In
this way, your secrets configuration details will be available in logs, but not
your password.
Secrets does not support using a static JAAS configuration file that
is passed at runtime, as is done with brokers.
Note
- The ZooKeeper Client JAAS configuration is not supported by secrets protection.
- You cannot encrypt multiple passwords in the same JAAS configuration using the Confluent CLI
secrets encryption command for encrypting the JAAS.
Before performing any operation on the JAAS configuration, you must first provide the
configuration key in the predefined path: <entry name>/<LoginModule>/<key>
.
Following is an example JAAS configuration in a property file (kafka/server.properties
):
sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required /
useKeyTab=false /
adminpassword=tempPass /
useTicketCache=true /
doNotPrompt=true;
Note
You can specify the path to adminpassword as: sasl.jaas.config/com.sun.security.auth.module.Krb5LoginModule/adminpassword
The standard CLI syntax for encrypting, adding, or updating a secrets JAAS
configuration is:
confluent secret file encrypt --config-file
--local-secrets-file
--remote-secrets-file
–-config
Of course, you must replace encrypt
with add
or update
, depending on
the configuration task you are performing.
This example shows the command and options for encrypting the password in a secrets JAAS configuration:
confluent secret file encrypt --config-file /etc/kafka/server.properties
--local-secrets-file /usr/secrets/security.properties
--remote-secrets-file /usr/secrets/security.properties
–-config sasl.jaas.config/com.sun.security.auth.module.Krb5LoginModule/adminpassword
This example shows the command and options for adding a new secrets JAAS configuration in which the password is encrypted:
confluent secret file add --config-file /etc/kafka/server.properties
--local-secrets-file /usr/secrets/security.properties
--remote-secrets-file /usr/secrets/security.properties
–-config sasl.jaas.config/com.sun.security.auth.module.Krb5LoginModule/adminpassword
This example shows the command and options for updating the encrypted password in the secrets JAAS configuration:
confluent secret file update --config-file /etc/kafka/server.properties
--local-secrets-file /usr/secrets/security.properties
--remote-secrets-file /usr/secrets/security.properties
–-config sasl.jaas.config/com.sun.security.auth.module.Krb5LoginModule/adminpassword
The encrypted data that you see after executing any of the preceding commands
should look similar to the following:
_metadata.master_key.0.salt = 3B++ViaAMaSwOOnxYI2bbeCtvZXRV5mxEOfb2FO3DnU=
_metadata.symmetric_key.0.created_at = 2020-03-06 14:07:34.248521 -0700 MST m=+0.011810143
_metadata.symmetric_key.0.envvar = CONFLUENT_SECURITY_MASTER_KEY
_metadata.symmetric_key.0.length = 32
_metadata.symmetric_key.0.iterations = 1000
_metadata.symmetric_key.0.salt = 4aqTrl8kdnQdVbGwkbeQHUiLA235/RKGC8zOXTHwQaI=
_metadata.symmetric_key.0.enc = ENC[AES/CBC/PKCS5Padding,data:bP93/lcsQVY5tzh4NWvD9tWO/yyTGwdAEgYHwpUokjiomma7QoH8X/jhlB7zibGd,iv:A7zk7hBwuataNy+ToT346w==,type:str]
server.properties/sasl.jaas.config/com.sun.security.auth.module.Krb5LoginModule/adminpassword = ENC[AES/CBC/PKCS5Padding,data:KdjkpudhWKoVe+6G35OYDw==,iv:5MgMsMT1o8d1JlXE0966Bg==,type:str]
Encrypt JSON configuration parameters
Secrets supports the encryption of JSON configuration parameters.
Before performing any operation on the JSON configuration parameters, you must
first provide the configuration key in the path format:
{
"name": "security configuration",
"credentials": {
"password": "password",
"ssl.keystore.location": "/usr/ssl"
}
}
You can specify the path to ssl.keystore.password
as:
<entry-name>.<key> :
credentials.password
credentials.ssl\\.keystore\\.location
The standard CLI syntax for encrypting, adding, updating, or decrypting a secrets JSON
configuration is:
confluent secret file encrypt --config-file
--local-secrets-file
--remote-secrets-file
–-config
Of course, you must replace encrypt
with add
, update
, or decrypt
,
depending on the configuration task you are performing.
This example shows the command and options for encrypting the password in the secrets JSON configuration:
./confluent secret file encrypt
--config-file /etc/kafka/sample.json
--local-secrets-file /usr/secrets/security.properties
--remote-secrets-file /usr/secrets/security.properties
--config credentials.password
Note
The --config
key for JSON configurations must be separated by .
in the path syntax.
This example shows the command and options for adding a new secrets JSON configuration in which the password is encrypted:
./confluent secret file add
--config-file /etc/kafka/sample.json
--local-secrets-file /usr/secrets/security.properties
--remote-secrets-file /usr/secrets/security.properties
--config credentials.password
This example shows the command and options for updating the encrypted password in a secrets JSON configuration:
./confluent secret file update
--config-file /etc/kafka/sample.json
--local-secrets-file /usr/secrets/security.properties
--remote-secrets-file /usr/secrets/security.properties
--config credentials.password
The encrypted data that you see after executing the preceding commands should
look similar to the following:
_metadata.master_key.0.salt = 3B++ViaAMaSwOOnxYI2bbeCtvZXRV5mxEOfb2FO3DnU=
_metadata.symmetric_key.0.created_at = 2020-03-06 14:07:34.248521 -0700 MST m=+0.011810143
_metadata.symmetric_key.0.envvar = CONFLUENT_SECURITY_MASTER_KEY
_metadata.symmetric_key.0.length = 32
_metadata.symmetric_key.0.iterations = 1000
_metadata.symmetric_key.0.salt = 4aqTrl8kdnQdVbGwkbeQHUiLA235/RKGC8zOXTHwQaI=
_metadata.symmetric_key.0.enc = ENC[AES/CBC/PKCS5Padding,data:bP93/lcsQVY5tzh4NWvD9tWO/yyTGwdAEgYHwpUokjiomma7QoH8X/jhlB7zibGd,iv:A7zk7hBwuataNy+ToT346w==,type:str]
server.properties/sasl.jaas.config/com.sun.security.auth.module.Krb5LoginModule/adminpassword = ENC[AES/CBC/PKCS5Padding,data:KdjkpudhWKoVe+6G35OYDw==,iv:5MgMsMT1o8d1JlXE0966Bg==,type:str]
sample.json/credentials.password = ENC[AES/CBC/PKCS5Padding,data:4cCPvtf9Sgpf6amU358NDw==,iv:Aq/OmYfGIdbyw78LRe5gHQ==,type:str]
This example shows the command and options for decrypting the password in a secrets JSON configuration:
./confluent secret file decrypt
--config-file /etc/kafka/sample.json
--local-secrets-file /usr/secrets/security.properties
--remote-secrets-file /usr/secrets/security.properties
--config credentials.password
Rotate keys
This command rotates the master key or data key.
Rotate master key (--master-key
): Generates a new master key and re-encrypts the data with the new master key. The new master
key is stored in an environment variable.
confluent secret file rotate --master-key \
--local-secrets-file /usr/secrets/security.properties \
-–passphrase @/User/bob/secret.properties
--passphrase-new @/User/bob/secretNew.properties
Rotate data key (--data-key
): Generates a new data key and re-encrypts the file with the new data key.
confluent secret file rotate --data-key \
--local-secrets-file /usr/secrets/security.properties \
-–passphrase @/User/bob/secret.properties
Script commands
You can script the commands by using stdin or from a file:
- To pipe from stdin use dash (
-
), for example --passphrase -
.
- To read from a file use
@<path-to-file>
, for example --passphrase @/User/bob/secret.properties
.
confluent secret master-key generate \
--local-secrets-file /usr/demo/security.properties \
--passphrase @/Users/user.name/tmp/demo/masterkey.properties
echo -e "demoMasterKey" | confluent secret master-key generate --local-secrets-file
/usr/demo/security.properties --passphrase -
Fix corrupt master key
If your master key becomes corrupt or is lost, you must create new passwords in the properties file (e.g.
ssl.keystore.password
) and generate a new master key for the encrypted configuration parameters. The following
example updates the ssl.keystore.password
in the /etc/kafka/server.properties
file.
Create a new password in your properties file.
ssl.keystore.password=mynewpassword
Generate a new master key.
confluent secret master-key generate \
--local-secrets-file <path-to-file>/security.properties \
--passphrase @/User/bob/secret.properties
Encrypt the configs with new master key.
confluent secret file encrypt --config-file /etc/kafka/server.properties \
--local-secrets-file <path-to-file>/security.properties \
--remote-secrets-file <path-to-file> \
--config ssl.keystore.password