ANSIBLE
Ansible Playbooks for Confluent Platform includes playbooks to upgrade all Confluent Platform components. The playbooks go host by host, shutting down the component, upgrading packages, and validating application health before moving onto the next one.
You must complete the upgrades in the following order:
The upgrade playbooks have the following requirements:
hosts.yml
Check out the branch of cp-ansible you want to upgrade to. Branches are in the format <version>-post, for example, 6.1.0-post.
cp-ansible
<version>-post
cd /path/to/cp-ansible git fetch git checkout 6.1.0-post
The ZooKeeper upgrade is designed to run on Kafka clusters with three nodes or more. As it upgrades the hosts it checks for quorum and health on each host before moving on to the next. Enter the command below to upgrade ZooKeeper:
ansible-playbook -i /path/to/hosts.yml upgrade_zookeeper.yml
The Kafka broker upgrade playbook upgrades each broker in sequence. The upgrade takes special care around the inter.broker.protocol.version and log.message.format.version properties.
inter.broker.protocol.version
log.message.format.version properties
Enter the following command and note the current version of confluent-kafka-2.12 or confluent-server installed on your hosts:
confluent-kafka-2.12
confluent-server
rpm -qa | grep confluent confluent-server-5.5.1-1.noarch
Run the upgrade_kafka_broker playbook and pass in the version installed:
upgrade_kafka_broker
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_broker.yml -e kafka_broker_upgrade_start_version=5.5.1
This playbook upgrades the Kafka brokers in sequence.
Important
The log.message.format.version property is not updated when the Kafka upgrade completes. This property is changed at the end of this procedure, after all clients have been upgraded.
log.message.format.version
Note
If you need to upgrade specific hosts instead of all of them, you can limit the upgrade. This can be useful when your components are behind a load balancer. In this case, remove a specific host from the load balancer pool, upgrade it, then add it back. This ensures no traffic is disrupted. Enter the following command to limit the upgrade to one or more specific hosts.
ansible-playbook -i /path/to/hosts.yml upgrade_<component>.yml --limit "<host1>,<host2>"
Use the commands below to update the other components:
Schema Registry
ansible-playbook -i /path/to/hosts.yml upgrade_schema_registry.yml
Connect
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_connect.yml
ksqlDB
ansible-playbook -i /path/to/hosts.yml upgrade_ksql.yml
The minimum version you can upgrade ksqlDB from is 5.5.0.
REST Proxy
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_rest.yml
Confluent Control Center
ansible-playbook -i /path/to/hosts.yml upgrade_control_center.yml
Confluent Platform Ansible playbooks do not currently support upgrading clients. Review the Confluent Platform upgrade preparation for additional information.
After all of the applicable clients have been updated, run the following command to update update the log format:
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_broker_log_format.yml -e kafka_broker_upgrade_end_version=6.1