CLOUD
Note
If you are installing the connector locally for Confluent Platform, see JDBC Connector (Source and Sink) for Confluent Platform.
The Kafka Connect MySQL Sink connector for Confluent Cloud exports data from Kafka topics to a MySQL database.
Important
After this connector becomes generally available, Confluent Cloud Enterprise customers will need to contact their Confluent Account Executive for more information about using this connector.
The MySQL Sink connector provides the following features:
auto.create
auto-evolve
kafka
none
record_value
You can manage your full-service connector using the Confluent Cloud API. For details, see the Confluent Cloud API documentation.
Refer to Cloud connector limitations for additional information.
Caution
Preview connectors are not currently supported and are not recommended for production use.
Use this quick start to get up and running with the Confluent Cloud MySQL sink connector. The quick start provides the basics of selecting the connector and configuring it to stream events to a MySQL database.
0.0.0.0/0
See the Quick Start for Apache Kafka using Confluent Cloud for installation instructions.
Click Connectors. If you already have connectors in your cluster, click Add connector.
Click the MySQL Sink connector icon.
Complete the following and click Continue.
INSERT
UPSERT
${topic}
kafka_${topic}
orders
kafka_orders
__connect_topic,__connect_partition,__connect_offset
3000
Configuration properties that are not shown in the Confluent Cloud UI use the default values. For default values and property definitions, see JDBC Sink Connector Configuration Properties.
Verify the connection details and click Launch.
The status for the connector should go from Provisioning to Running.
Verify that new records are being added to the MySQL database.
Tip
When you launch a connector, a Dead Letter Queue topic is automatically created. See Dead Letter Queue for details.
See also
For an example that shows fully-managed Confluent Cloud connectors in action with Confluent Cloud ksqlDB, see the Cloud ETL Demo. This example also shows how to use Confluent Cloud CLI to manage your resources in Confluent Cloud.
Complete the following steps to set up and run the connector using the Confluent Cloud CLI.
Make sure you have all your prerequisites completed.
Enter the following command to list available connectors:
ccloud connector-catalog list
Enter the following command to show the required connector properties:
ccloud connector-catalog describe <connector-catalog-name>
For example:
ccloud connector-catalog describe MySqlSink
Example output:
Following are the required configs: connector.class: MySqlSink input.data.format name kafka.api.key kafka.api.secret connection.host connection.port connection.user connection.password db.name tasks.max topics
Create a JSON file that contains the connector configuration properties. The following example shows required and optional connector properties:
{ "topics": "pageviews", "input.data.format": "AVRO", "connector.class": "MySqlSink", "name": "MySqlSinkConnector_0", "kafka.api.key": "****************", "kafka.api.secret": "****************************************************************", "connection.host": "dev-testing-temp.dvomdiaryntj.us-west-7.rds.amazonaws.com", "connection.port": "3306", "connection.user": "admin", "connection.password": "**********", "db.name": "test", "insert.mode": "INSERT", "auto.create": "true", "auto.evolve": "true", "tasks.max": "1" }
Note the following property definitions:
"topics"
"input.data.format"
"connector.class"
"name"
"insert.mode"
"auto.create"
"auto-evolve"
false
"tasks.max"
Optional:
"pk.mode"
"pk.fields"
pk.mode
Enter the following command to load the configuration and start the connector:
ccloud connector create --config <file-name>.json
ccloud connector create --config mysql-server-sink-config.json
Created connector MySqlSinkConnector_0 lcc-ix4dl
Enter the following command to check the connector status:
ccloud connector list
ID | Name | Status | Type +-----------+-------------------------+---------+------+ lcc-ix4dl | MySqlSinkConnector_0 | RUNNING | sink