Kafka Endpoints Reference

This is a reference for the Kafka Endpoints integration. The following properties are unique to this integration.

The endpoint configuration follows the same pattern as the Kafka moogdb module; there is a minimum set of required parameters and an optional set of parameters. See the descriptions below for detailed descriptions.

See the Apache Kafka documentation for details on SSL and SASL in Kafka.

Name

A unique name for the endpoint for use in Workflow Engine configuration.

Type	String
Required	Yes
Default	N/A

Servers

The list of brokers to connect to. A server configuration consists of the server name and port.

Server Name

Name of the server to connect to.

Type	String
Required	Yes
Default	N/A

Port

Port to communicate over.

Type	Integer
Required	Yes
Default	`9092`

Compression

The compression algorithm to use.

Type	One of: `none`, `lz4`, `gzip`, `snappy`.
Required	No
Default	`none`

SSL Configuration

See the Apache Kafka documentation for detailed descriptions of SSL configuration in Kafka.

USESSL

Whether to use SSL configuration. Check the box to enable.

Type	Boolean
Required	No
Default	Disabled

truststore.location

Path to the SSL truststore file.

Type	String
Required	Yes, if using SSL.
Default	N/A

truststore.password

Password for the SSL truststore file.

Type	String
Required	Yes, if using SSL.
Default	N/A

keystore.location

Path to the keystore file.

Type	String
Required	Yes, if using SSL.
Default	N/A

keystore.password

Password for the keystore file.

Type	String
Required	Yes, if using SSL.
Default	N/A

key.password

SSL certificate password.

Type	String
Required	Yes, if using SSL.
Default	N/A

endpoint.identification.algorithm

Method to validate server hosts, for example HTTPS. Leave blank to disable.

Type	String
Required	Yes, if using SSL.
Default	N/A

SASL Configuration

See the Apache Kafka documentation for detailed descriptions of SASL configuration in Kafka.

USESASL

Whether to use SASL. Check the box to enable.

Type	Boolean
Required	No
Default	Disabled

sasl.mechanism

SASL mechanism to use to authenticate.

Type	Choose from `PLAIN`, `SCRAM-SHA-256`, `SCRAM-SHA-512`, `OAUTHBEARER`
Required	Yes, if using SASL.
Default	N/A

security.protocol

Security protocol to use.

Type	Choose from `SASL_SSL`, `SASL_PLAINTEXT`
Required	Yes, if using SASL.
Default	N/A

sasl.jaas.config

Base module to use. Each requires additional parameters, defined under Additional SASL JaaS Config.

Type	One of: `PLAIN`, `SCRAM`, `OAUTHBEARER`, `GSSAPI`
Required	Yes, if using SASL.
Default	N/A

Additional SASL JaaS Config

Each JaaS base module (PLAIN, SCRAM, OAUTHBEARER, GSSAPI) requires additional parameters which you specify in this field.

The UI creates the string up to and including "required". This field must contain the remainder of the string after this. For example, if the connection was using "PLAIN", the "Additional SASL JaaS config" should contain the username and password:

username=John.Doe password=PASSWORD123

Example configurations for each base module are as follows:

Type	Example
PLAIN	`org.apache.kafka.common.security.plain.PlainLoginModule required username=Jane.Doe password=Password123;`
SCRAM	`org.apache.kafka.common.security.scram.ScramLoginModule required username=John.Doe password=My_Password;`
OAUTHBEARER	`org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required unsecuredLoginPrincipalClaimName="sub" unsecuredLoginStringClaim_sub="Jane.Doe";`
GSSAPI	`com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="KEYTAB_FILE_PATH_HERE" principal="SERVICENAME/HOST@REALM";`

Additional Properties

SASL Login refresh parameters. You only need to configure these if sasl.jaas.config is set to OAUTHBEARER. Defaults to Kafka recommended default values.

window.factor

Login refresh thread will sleep until the specified window factor relative to the credential's lifetime is reached, at which point it attempts to refresh the credential.

Type	Double
Required	Yes, if `sasl.jaas.config` is set to `OAUTHBEARER`.
Default	`0.8`
Valid Values	Between `0.5` (50%) and `1.0` (100%) inclusive.

window.jitter

Maximum amount of random jitter relative to the credential's lifetime that is added to the login refresh thread's sleep time. Legal values are between 0 and 0.25 (25%) inclusive.

Type	Double
Required	Yes, if `sasl.jaas.config` is set to `OAUTHBEARER`.
Default	Between `0` and `0.25` (25%) inclusive.

min.period.seconds

Desired minimum time for the login refresh thread to wait before refreshing a credential, in seconds. This value and sasl.login.refresh.buffer.seconds are both ignored if their sum exceeds the remaining lifetime of a credential.

Type	Short
Required	Yes, if `sasl.jaas.config` is set to `OAUTHBEARER`.
Default	`60`
Valid Values	`0` and `900`

min.buffer.seconds

When refreshing a credential, amount of buffer time to maintain, in seconds, before credential expiration. If a refresh would otherwise occur closer to expiration than the number of buffer seconds, the refresh is moved up to maintain as much of the buffer time as possible. This value and sasl.login.refresh.min.period.seconds are both ignored if their sum exceeds the remaining lifetime of a credential.

Type	Short
Required	Yes, if `sasl.jaas.config` is set to `OAUTHBEARER`.
Default	`300`
Valid Values	Between `0` and `3600`

Kerberos Properties

You only need to configure these if sasl.jaas.config is set to GSSAPI.

kerberos.debug.log

Whether to use authentication debugging. Check to enable.

Type	Boolean
Required	Yes, if `sasl.jaas.config` is set to `GSSAPI`.
Default	Disabled

sasl.kerberos.service.name

Name of the Kerberos service.

Type	String
Required	Yes, if `sasl.jaas.config` is set to `GSSAPI`.
Default	N/A

kerberos.conf.file.path

Path to the Kerberos configuration file.

Type	String
Required	Yes, if `sasl.jaas.config` is set to `GSSAPI`.
Default	N/A

In this section: