If you encountered any issues while going through the quickstart workflow, review the following resolutions before trying the steps again.
Issue: Cannot locate the Datagen connector
Resolution: Verify that you have added the location of the Confluent Platform bin
directory to your PATH
as described in Step 1: Download and Start Confluent Platform:
export PATH=<path-to-confluent>/bin:$PATH
Resolution: Verify the DataGen connector is installed and running.
Ensure that the kafka-connect-datagen
is installed and running as described in Step 1: Download and Start Confluent Platform.
confluent-hub install --no-prompt confluentinc/kafka-connect-datagen:latest
Your output should resemble:
Running in a "--no-prompt" mode
...
Completed
Resolution: Check the connect logs for Datagen
using the Confluent CLI confluent local services connect log command.
confluent local services connect log | grep -i Datagen
Your output should resemble:
[2019-04-18 14:21:08,840] INFO Loading plugin from: /Users/user.name/Confluent/confluent-version/share/confluent-hub-components/confluentinc-kafka-connect-datagen (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:215)
[2019-04-18 14:21:08,894] INFO Registered loader: PluginClassLoader{pluginLocation=file:/Users/user.name/Confluent/confluent-version/share/confluent-hub-components/confluentinc-kafka-connect-datagen/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:238)
[2019-04-18 14:21:08,894] INFO Added plugin 'io.confluent.kafka.connect.datagen.DatagenConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:167)
[2019-04-18 14:21:09,882] INFO Added aliases 'DatagenConnector' and 'Datagen' to plugin 'io.confluent.kafka.connect.datagen.DatagenConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:386)
Resolution: Verify the .jar
file for kafka-connect-datagen
has been added and is present in the lib
subfolder.
ls $CONFLUENT_HOME/share/confluent-hub-components/confluentinc-kafka-connect-datagen/lib/
Your output should resemble:
...
kafka-connect-datagen-0.1.0.jar
...
Resolution: Verify the plugin exists in the connector path.
When you installed the kafka-connect-datagen
file from Confluent hub, the installation directory
is added to the plugin path of several properties files:
Adding installation directory to plugin path in the following files:
/Users/user.name/Confluent/confluent-version/etc/kafka/connect-distributed.properties
/Users/user.name/Confluent/confluent-version/etc/kafka/connect-standalone.properties
/Users/user.name/Confluent/confluent-version/etc/schema-registry/connect-avro-distributed.properties
/Users/user.name/Confluent/confluent-version/etc/schema-registry/connect-avro-standalone.properties
...
You can use any of them to check the
connector path. This example uses the connect-avro-distributed.properties
file.
grep plugin.path $CONFLUENT_HOME/etc/schema-registry/connect-avro-distributed.properties
Your output should resemble:
plugin.path=share/java,/Users/user.name/Confluent/confluent-version/share/confluent-hub-components
Confirm its contents are present:
ls $CONFLUENT_HOME/share/confluent-hub-components/confluentinc-kafka-connect-datagen
Your output should resemble:
assets doc lib manifest.json
Issue: Stream-Stream joins error
An error states Stream-Stream joins must have a WITHIN
clause specified. This error can occur if you created both pageviews
and users
as streams by mistake.
Resolution: Ensure that you created a stream for pageviews
, and a table for users
in Step 4: Create and Write to a Stream and Table using ksqlDB.