Configuring the REST API for HTTP or HTTPS
By default Schema Registry allows clients to make REST API calls over HTTP. You may configure Schema Registry to allow either HTTP or HTTPS or both at the same time.
The following configuration determines the protocol used by Schema Registry:
listeners
Comma-separated list of listeners that listen for API requests over HTTP or HTTPS or both. If a listener uses HTTPS, the appropriate SSL configuration parameters need to be set as well.
On the clients, configure schema.registry.url
to match the configured Schema Registry listener.
Additional configurations for HTTPS
If you configure an HTTPS listener, there are several additional configurations for Schema Registry.
First, configure the appropriate SSL configurations for the keystore and optionally truststore for the Schema Registry cluster (for example, in schema-registry.properties
). The truststore is required only when ssl.client.auth
is set to true.
ssl.truststore.location=/etc/kafka/secrets/kafka.client.truststore.jks
ssl.truststore.password=<password>
ssl.keystore.location=/etc/kafka/secrets/kafka.client.keystore.jks
ssl.keystore.password=<password>
ssl.key.password=<password>
You may specify which protocol to use while making calls between the instances of Schema Registry. The secondary to primary node calls for writes and deletes will use the specified protocol.
inter.instance.protocol
The protocol used while making calls between the instances of Schema Registry. The secondary to primary node calls for writes and deletes will use the specified protocol. The default value is http
. When https
is set, ssl.keystore.
and ssl.truststore.
configs are used while making the call. The schema.registry.inter.instance.protocol
name is deprecated; use inter.instance.protocol
instead.
- Type: string
- Default: “http”
- Importance: low
Starting with 5.4, Confluent Platform provides the Schema Registry dedicated client configuration
properties, as shown in the example.
Tip
Clients to Schema Registry include both:
- Client applications created or used by developers.
- Confluent Platform components such as Confluent Control Center, Kafka Connect, ksqlDB, and so forth.
To configure clients to use HTTPS to Schema Registry, set the following properties or environment variables:
On the client, configure the schema.registry.url
to match the configured listener for HTTPS.
<client>.schema.registry.url: "<schema-registry-url>:<port>"
On the client, configure the environment variables to set the SSL keystore and truststore in one of two ways:
(Recommended) Use the Schema Registry dedicated properties to configure the client:
schema.registry.ssl.truststore.location=/etc/kafka/secrets/kafka.client.truststore.jks
schema.registry.ssl.truststore.password=<password>
schema.registry.ssl.keystore.location=/etc/kafka/secrets/kafka.client.keystore.jks
schema.registry.ssl.keystore.password=<password>
schema.registry.ssl.key.password=<password>
The naming conventions for Confluent Control Center configuration differ slightly from the other clients. To configure Control Center as an HTTPS client to Schema Registry, specify these dedicated properties in the Control Center config file:
confluent.controlcenter.schema.registry.schema.registry.ssl.truststore.location=/etc/kafka/secrets/kafka.client.truststore.jks
confluent.controlcenter.schema.registry.schema.registry.ssl.truststore.password=<password>
confluent.controlcenter.schema.registry.schema.registry.ssl.keystore.location=/etc/kafka/secrets/kafka.client.keystore.jks
confluent.controlcenter.schema.registry.schema.registry.ssl.keystore.password=<password>
confluent.controlcenter.schema.registry.schema.registry.ssl.key.password=<password>
(Legacy, on client) Set environment variables depending on the client (one of KAFKA_OPTS
, SCHEMA_REGISTRY_OPTS
, KSQL_OPTS
):
export JAVA_OPTS: "-Djavax.net.ssl.trustStore=/etc/kafka/secrets/kafka.client.truststore.jks \
-Djavax.net.ssl.trustStorePassword=<password> \
-Djavax.net.ssl.keyStore=/etc/kafka/secrets/kafka.client.keystore.jks \
-Djavax.net.ssl.keyStorePassword=<password>"
Important
If you are using broker schema validation, you must use the environment variables on Confluent Server, as dedicated Schema Registry properties are not available on the brokers.
If you use the legacy method of defining SSL values in system environment variables, SSL settings will apply to
every Java component running on this JVM. For example on Connect, every connector
will use the given truststore. Consider a scenario where you are using an Amazon Web Servces (AWS) connector such
as S3 or Kinesis, and do not have the AWS certificate chain in the given truststore. The connector will fail with
the following error:
com.amazonaws.SdkClientException: Unable to execute HTTP request:
sun.security.validator.ValidatorException: PKIX path building failed
This does not apply if you use the dedicated Schema Registry client configurations.
For the kafka-avro-console-consumer
and kafka-avro-console-consumer
, you must pass the Schema Registry properties on the command line. Here is an example for the producer:
./kafka-avro-console-producer --broker-list localhost:9093 --topic myTopic \
--producer.config ~/ect/kafka/producer.properties --property value.schema=‘{“type”:“record”,“name”:“myrecord”,“fields”:[{“name”:“f1”,“type”:“string”}]}’ \
--property schema.registry.url=https://localhost:8081 --property schema.registry.ssl.truststore.location=/etc/kafka/security/schema.registry.client.truststore.jks --property schema.registry.ssl.truststore.password=myTrustorePassword
For more examples of using the producer and consumer command line utilities, see Test Drive Avro Schema, Test Drive JSON Schema, Test Drive Protobuf Schema, and the demo in Schema Validation.
Migrating from HTTP to HTTPS
To upgrade Schema Registry to allow REST API calls over HTTPS in an existing cluster:
- Add/Modify the
listeners
config to include HTTPS. For example: http://0.0.0.0:8081,https://0.0.0.0:8082
- Configure Schema Registry with appropriate SSL configurations to setup the keystore and optionally truststore
- Do a rolling bounce of the cluster
This process enables HTTPS, but still defaults to HTTP so Schema Registry instances can still communicate before all nodes have been restarted. They will continue to use HTTP as the default until configured not to. To switch to HTTPS as the default and disable HTTP support, perform the following steps:
- Enable HTTPS as mentioned in first section of upgrade (both HTTP & HTTPS will be enabled)
- Configure
inter.instance.protocol
to https in all the nodes
- Do a rolling bounce of the cluster
- Remove http listener from the
listeners
in all the nodes
- Do a rolling bounce of the cluster
Configuring the REST API for Basic HTTP Authentication
Schema Registry can be configured to require users to authenticate using a username and password via the Basic HTTP authentication mechanism.
Use the following settings to configure Schema Registry to require authentication:
authentication.method=BASIC
authentication.roles=<user-role1>,<user-role2>,...
authentication.realm=<section-in-jaas_config.file>
The authentication.roles
config defines a comma-separated list of user roles. To be authorized
to access Schema Registry, an authenticated user must belong to at least one of these roles.
For example, if you define admin
, developer
, user
, and sr-user
roles,
the following configuration assigns them for authentication:
authentication.roles=admin,developer,user,sr-user
The authentication.realm
config must match a section within jaas_config.file
, which
defines how the server authenticates users and should be passed as a JVM option during server start:
export SCHEMA_REGISTRY_OPTS=-Djava.security.auth.login.config=/path/to/the/jaas_config.file
<path-to-confluent>/bin/schema-registry-start <path-to-confluent>/etc/schema-registry/schema-registry.properties
An example jaas_config.file
is:
SchemaRegistry-Props {
org.eclipse.jetty.jaas.spi.PropertyFileLoginModule required
file="/path/to/password-file"
debug="false";
};
Assign the SchemaRegistry-Props
section to the authentication.realm
config setting:
authentication.realm=SchemaRegistry-Props
The example jaas_config.file
above uses the Jetty PropertyFileLoginModule
, which
authenticates users by checking for their credentials in a password file.
You can also use other implementations of the standard Java LoginModule
interface, such as
the LdapLoginModule
, or the JDBCLoginModule
for reading credentials from a database.
The file parameter is the location of the password file. The format is:
<username>: <password-hash>,<role1>[,<role2>,...]
Here’s an example:
fred: OBF:1w8t1tvf1w261w8v1w1c1tvn1w8x,user,admin
barney: changeme,user,developer
betty: MD5:164c88b302622e17050af52c89945d44,user
wilma: CRYPT:adpexzg3FUZAk,admin,sr-user
Get the password hash for a user by using the org.eclipse.jetty.util.security.Password
utility:
bin/schema-registry-run-class org.eclipse.jetty.util.security.Password fred letmein
Your output should resemble:
letmein
OBF:1w8t1tvf1w261w8v1w1c1tvn1w8x
MD5:0d107d09f5bbe40cade3de5c71e9e9b7
CRYPT:frd5btY/mvXo6
Each line of the output is the password encrypted using different mechanisms, starting with
plain text.
Once Schema Registry is configured to use Basic authentication, clients must be
configured with suitable valid credentials, for example:
basic.auth.credentials.source=USER_INFO
basic.auth.user.info=fred:letmein
Tip
The schema.registry
prefixed versions of these properties were deprecated in Confluent Platform 5.0.
schema.registry.basic.auth.credentials.source
is deprecated.
schema.registry.basic.auth.user.info
is deprecated.