Overview
The Confluent Telemetry Reporter is a plugin that runs inside each Confluent Platform service to
push metadata about the service to Confluent. Telemetry Reporter enables product features
based on the metadata, like Proactive Support. Data is sent over HTTP using an encrypted connection.
Important
No data is sent to Confluent Cloud if you set confluent.telemetry.enabled
to false
.
Installation
Confluent Telemetry Reporter is installed as part of the full Confluent Platform installation. If you choose to
install packages individually, ensure that Telemetry Reporter is installed by using
yum install confluent-telemetry
.
Enable telemetry reporting to Confluent Cloud
- Prerequisites
-
Create a Cloud API key to authenticate with Confluent Cloud. For example:
ccloud login
ccloud api-key create --resource cloud
Note
You must use a Cloud API Key. Using the Cluster API Key that is used to communicate with Kafka causes an authentication error.
Configuration with Ansible Playbooks for Confluent Platform
If you use Ansible Playbooks for Confluent Platform to manage your Confluent Platform installation, add the following configuration overrides
to all Confluent Platform roles and deploy the changes.
telemetry_enabled: true
telemetry_api_key: <api-key>
telemetry_api_secret: <api-secret>
Configuration with Operator
When you use Confluent Operator to manage Confluent Platform, you can enable telemetry as described in
this section.
Enable telemetry at the global or component level by setting the following
in the Operator configuration file ($VALUES_FILE
):
telemetry:
enabled: true --- [1]
secretRef: <confluent-telemetry-secret> --- [2]
proxy: false --- [3]
- [1] Set
enabled: true
to enable telemetry.
- [2] Set
secretRef
to the Kubernetes secret that contains the key
telemetry
. You create this secret (noted as
<confluent-telemetry-secret>
) as described in the last step below.
- [3] Set
proxy: true
to enable routing telemetry data through an
authenticated HTTP proxy. If proxy
is set to true
, then the secret in
telemetry.secretRef
must additionally contain proxy configuration.
Create the telemetry.txt
file, and add the following in the file:
apiKey=<ccloud-api-key>
apiSecret=<ccloud-api-secret>
If you set telemetry.proxy: true
in the first step above, add the
following in the telemetry.txt
file:
proxyUrl=<proxyUrl>
proxyUsername=<username>
proxyPassword=<password>
Create a Kubernetes secret with the name you provided in
telemetry.secreteRef
in the first step ([2]).
<confluent-telemetry-secret>
is used as an example in this section.
kubectl --namespace <namespace> create secret generic \
<confluent-telemetry-secret> \
--from-file=telemetry=<path>/telemetry.txt
Custom Deployments Configuration
For all Confluent Platform components other than Confluent Server, enable telemetry reporting by adding the following
lines to the configuration file for the service and restarting the service to deploy the
new configuration. Replace <API_KEY>
and <API_SECRET>
with the key and secret generated above.
metric.reporters=io.confluent.telemetry.reporter.TelemetryReporter
confluent.telemetry.enabled=true
confluent.telemetry.api.key=<API_KEY>
confluent.telemetry.api.secret=<API_SECRET>
For Confluent Server, the metric.reporters
configuration is not needed. Add the following lines to
the Confluent Server configuration file and restart Confluent Server. If restarting Confluent Server is undesirable, you can
add these configurations by using dynamic configuration and the kafka-config
CLI.
confluent.telemetry.enabled=true
confluent.telemetry.api.key=<API_KEY>
confluent.telemetry.api.secret=<API_SECRET>
Configuration with a proxy
Confluent Telemetry Reporter supports routing telemetry data through an authenticated HTTP proxy.
In these environments, set the followingparameters to have telemetry data egress through the proxy.
confluent.telemetry.proxy.url=http://<proxy_server_ip>:<proxy_server_port>
confluent.telemetry.proxy.username=<proxy-username>
confluent.telemetry.proxy.password=<proxy-password>
What data is sent and how it is used
When a customer has Confluent Telemetry Reporter enabled for a specific Confluent Platform component,
Confluent collects metrics about the Confluent Platform component
that include runtime performance metrics, the Kafka version, the Confluent Platform version,
a unique identifier that’s valid for the applicable Confluent Platform component,
a unique identifier for the Kafka cluster, and a unique identifier for the Customer organization.
The data is sent using an encrypted connection to Confluent servers once per minute by default.
The resulting data is used by Confluent to offer Proactive Support to its customers and to maintain
and improve Confluent Products and Services.For this data to be sent, enable the transmission by
setting configuration flags in every Confluent Platform component. You can stop sending data at any time by
removing these configuration flags.
Dynamic Configuration
Note
Currently, dynamic configuration applies only to Kafka brokers.
To understand how to apply dynamic configs for your cluster, see
Kafka Dynamic Configurations.
Enable Telemetry without a restart
If you’re using Confluent Server, you can enable the Telemetry Reporter without a restart
by using the following command. Replace <API_KEY>
and <API_SECRET>
with
the key and secret generated above.
./bin/kafka-configs --bootstrap-server localhost:9092 --alter --entity-type brokers --entity-default --add-config confluent.telemetry.enabled=true,confluent.telemetry.api.key=<API_KEY>,confluent.telemetry.api.secret=<API_SECRET>
Supported dynamic configuration properties
confluent.telemetry.enabled
confluent.telemetry.api.key
confluent.telemetry.api.secret
confluent.telemetry.proxy.url
confluent.telemetry.proxy.username
confluent.telemetry.proxy.password