CLOUD
Note
Preview Note
Follow this procedure to configure Azure Private Link for a Dedicated cluster in Azure.
Warning
For limitations of the Azure Private Link feature, see Limitations.
To make an Azure Private Link connection to a cluster in Confluent Cloud you must register the Azure subscription ID you wish to use. This is a security measure that enables Confluent to ensure that only your organization can initiate Azure Private Link connections to the cluster. Azure Private Link connections from a VNET not contained in a registered Azure subscription will not be accepted by Confluent Cloud.
After the connection status is “Active” in the Confluent Cloud UI, you must configure Private Endpoint(s) in your VNET from Azure portal to make the Private Link connection to your Confluent Cloud cluster.
In the Confluent Cloud UI you will find the following information for your Confluent Cloud cluster under the Cluster Settings section. This information is needed to configure Azure Private Link for a Dedicated cluster in Azure.
Create the following Private Endpoint(s) through the Azure Private Link Center:
DNS changes must be made to ensure connectivity passes through Azure Private Link in the supported pattern. Any DNS provider that can ensure DNS is routed as follows is acceptable. Azure Private DNS Zone (used in this example) is one option.
Update DNS using Azure Private DNS Zone in the Azure console:
Create the Private DNS Zone.
Search for the Private DNS Zone resource in Azure portal.
Click Add
Copy the DNS Domain name from the Networking tab under Cluster Settings in the Confluent Cloud UI and use it as the name for the Private DNS Zone.
For example:
4kgzg.centralus.azure.confluent.cloud
Notice there is no glb in the DNS Domain name
Fill in subscription, resource group and name and click Review + create.
Wait for the Azure deployment to complete.
Create DNS records.
Attach the Private DNS Zone to the VNET(s) where clients/applications are present.
From an instance within the VNET (or anywhere the previous step’s DNS is set up), run the following to validate Kafka connectivity through Azure Private Link is working correctly.
Set a variable with the cluster bootstrap URL.
export BOOTSTRAP=$ConfluentCloudBootstrap
export BOOTSTRAP=lkc-222v1o-4kgzg.centralus.azure.glb.confluent.cloud:9092
Test connectivity to the cluster.
openssl s_client -connect $BOOTSTRAP:9092 -servername $BOOTSTRAP -verify_hostname $BOOTSTRAP </dev/null 2>/dev/null | grep -E 'Verify return code|BEGIN CERTIFICATE' | xargs
If the return output is -----BEGIN CERTIFICATE----- Verify return code: 0 (ok), connectivity to the bootstrap is confirmed.
-----BEGIN CERTIFICATE----- Verify return code: 0 (ok)
You might need to update the network security tools and firewalls to allow connectivity. If you have issues connecting after following these steps, confirm which network security systems your organization uses and whether their configurations need to be changed.
Next, verify connectivity with the Confluent Cloud CLI.
Log in to the Confluent Cloud CLI with your Confluent Cloud credentials.
ccloud login
List the clusters in your organization.
ccloud kafka cluster list
Select the cluster with Azure Private Link you wish to test.
ccloud kafka cluster use ...
ccloud kafka cluster use lkc-222v1o
Create a cluster API key to authenticate with the cluster.
ccloud api-key create --resource ... --description ...
ccloud api-key create --resource lkc-222v1o --description "connectivity test"
Select the API key you just created.
ccloud api-key use ... --resource ...
ccloud api-key use R4XPKKUPLYZSHOAT --resource lkc-222v1o
Create a test topic.
ccloud kafka topic create test
Start consuming events from the test topic.
ccloud kafka topic consume test
Open another terminal tab or window.
Start a producer.
ccloud kafka topic produce test
Type anything into the produce tab and hit Enter; press Ctrl+D or Ctrl+C to stop the producer.
Enter
Ctrl+D
Ctrl+C
The tab running consume will print what was typed in the tab running produce.
You’re done! The cluster is ready for use.