Encrypt Confluent Cloud clusters using self-managed keys – Amazon Web Services

When you create a Confluent Cloud Dedicated cluster on Amazon Web Services, you can optionally use self-managed encryption keys to protect data at rest, allowing only the appropriate entity or user can decrypt it. Also known as bring your own key (BYOK) encryption, self-managed keys provide you greater privacy and data integrity, which is frequently required by government, health, finance, and many other industries.

Protect the data at rest stored in your Confluent Cloud Dedicated clusters on Amazon Web Services using AWS Key Management Service (KMS) to create and manage encryption keys. Use of self-managed encryption keys for clusters in Confluent Cloud on Amazon Web Services includes the following operations and limitations:

  • Create Dedicated clusters that use self-managed encryption keys during cluster creation. You cannot switch between Automatic (default) and Self-managed encryption modes after the cluster has been provisioned.
  • Use AWS Key Management Service (KMS) to generate, use, rotate, and destroy your encryption keys.
  • Customer master keys (CMKs) are supported.
  • The encryption key used must match the zone specified for the cluster.
  • Only symmetric keys are supported.
  • Importing key material is not supported.
  • Revoke encryption keys to prevent access to stored data.
  • Enable automatic key rotation using AWS KMS. Manual key rotation is not supported.
  • Manually-provisioned clusters, including the use of AWS Transit Gateway and some networking configuration edge cases, are not supported.

Create a Dedicated cluster with self-managed encryption

Warning

If you accidentally delete the master key, you will no longer be able to access your encrypted data. Neither Confluent nor AWS can regain access to your data.

To create an encrypted Confluent Cloud Dedicated cluster on Amazon Web Services that uses a self-managed encryption key:

  1. Navigate to the Clusters page for your environment and click Create cluster if you are creating the first cluster in your environment, or click ** Add cluster** if other clusters exist.

  2. For 1. Select cluster type under Create cluster, select Dedicated and click Begin Configuration.

  3. For 2. Regions/zones under Create cluster, select AWS as the cloud service provider, select the Region and Availability, and then click Continue.

  4. For 3. Networking under Create cluster, select the networking type and click Continue.

  5. For 4 - Security under Create cluster, select Self-managed to manage your own encryption key using AWS Key Management Service. Additional steps appear.

    Step 1: Enter the Amazon Resource Name (ARN) for your encryption key. To locate your ARN, log in to the AWS KMS Console and create or locate the ARN.

    Step 2: You must authorize your AWS key policy to include authorization access for Confluent. Copy and append the permissions provided by Confluent Cloud into your AWS KMS key policy, then click Continue. This authorizes Confluent access to your KMS. For details, see the AWS KMS documentation.

    Note

    After you complete the cluster creation process, this cluster-key pairing is locked. You cannot change it for the lifetime of the cluster. You can still modify permissions related to the key, and also disable or delete it, as long as your AWS permissions allow for it.

  6. For 5. Review and launch under Create cluster. enter the Cluster name and click Launch Cluster.

Note

A successful validation results in the provisioning of your cluster. If the cluster configuration is invalid because the encryption key is not valid or not authorized for Confluent, then you will get an error message indicating so. Close the modal; any invalid fields will be highlighted in the original form. Reenter a valid value in the highlighted field.

Using the Confluent Cloud CLI to encrypt clusters using your own key

As mentioned above, you must use AWS as the provider, and specify the dedicated cluster type.

The following Confluent Cloud CLI example shows how to create an encrypted Confluent Cloud cluster using your customer-managed key. Any content in brackets (<>) must be customized for your environment.

ccloud kafka cluster create sales092020 --cloud "aws" --region "<KMS-region>" --type "dedicated" --cku <CKU-value> --encryption-key "<AWS-ARN-ID>"

Copy and append these permissions to the existing "Statements" array field in the key policy of your ARN to authorize access for Confluent:

{
    "Sid" : "Allow Confluent account (123456789101) to use the key",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : ["arn:aws:iam::123456789101:root"]
    },
    "Action" : [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ],
    "Resource" : "*"
  }, {
    "Sid" : "Allow Confluent account (123456789101) to attach persistent resources",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : ["arn:aws:iam::123456789101:root"]
    },
    "Action" : [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ],
    "Resource" : "*"
}

Please confirm you've authorized the key for this account: 123456789101 (y/n):y

When you specify the --encryption-key option, you are prompted to update your AWS KMS policy.

For details, see:

View the Dedicated cluster security settings

You can view the Security settings for Dedicated clusters provisioned on AWS. In other words, if you used Automatic, Self-managed, or have an existing Dedicated cluster on AWS that you created prior to using the Self-managed encryption option, you can view the cluster security settings. The data in the cluster security settings is informational only and serves to identify the keys in use.

To view your dedicated AWS cluster security settings:

  1. Select your Confluent Cloud cluster.
  2. Click the Cluster settings tab and then Security.

Note that anyone authorized to view your AWS Dedicated clusters can view this data.