Skip to main content

Kafka Forwarder Integration Troubleshooting

Common Issues

Issue: Kafka Connection Fails

Symptoms: Integration is inactive, or connection errors appear in the logs.

Cause: Incorrect broker addresses, network connectivity issues, or firewall restrictions.

Solution:

  1. Verify that the broker hostnames and ports are correct.

  2. Test network connectivity by using telnet or nc.

  3. Check that firewall rules allow traffic to Kafka ports.

  4. Ensure that the Kafka brokers are running and accessible.

Issue: SASL Authentication Errors

Symptoms: Authentication-failed messages appear in the logs.

Cause: Incorrect credentials, an incorrect SASL mechanism, or JAAS configuration errors.

Solution:

  1. Verify that the username and password are correct.

  2. Ensure that the SASL mechanism matches the Kafka configuration.

  3. Validate the JAAS configuration string syntax.

  4. For Kerberos authentication, verify the service name and the krb5.conf configuration file.

Issue: SSL Certificate Errors

Symptoms: SSL handshake failures or certificate validation errors occur.

Cause: Incorrect certificate paths, expired certificates, or truststore configuration issues.

Solution:

  1. Verify that the truststore and keystore file paths exist.

  2. Check that the certificates have not expired.

  3. Ensure that the truststore contains the correct CA certificates.

  4. Validate that the keystore contains the appropriate private key.

Issue: Messages Do Not Arrive in Kafka

Symptoms: No errors appear in the logs, but messages do not reach Kafka topics.

Cause: Missing topics, incorrect topic names, or insufficient permissions.

Solution:

  1. Verify that the topics exist in the Kafka cluster.

  2. Ensure that the topic names match exactly, including letter case.

  3. Confirm that Moogsoft Onprem has permission to write to the topics.

  4. Check the topic partition and replication configuration.

Issue: Transformation Rules Do Not Work

Symptoms: Empty payloads, missing fields, or incorrect data appears in Kafka.

Cause: Incorrect macro syntax, missing fields, or rule configuration errors.

Solution:

  1. Verify that the macro syntax uses the correct field names.

  2. Ensure that the field names exist in the source data.

  3. Test the transformation rules with sample data.

  4. Review the filter expressions for correct syntax.

Error Messages

Error Message

Cause

Resolution

Unable to load KafkaForwarder configuration from integration

Integration is not configured correctly or is missing.

Verify that the integration is installed and configured correctly.

Unable to parse KafkaForwarder integration config

Invalid JSON or configuration format.

Check the integration configuration syntax.

Failed to find the requested Kafka connection

The connection name does not exist or is misspelled.

Verify that the connection name matches a configured connection.

Expected to see a list of servers in the connection config

Missing or invalid server configuration.

Ensure that Kafka broker servers are configured correctly.

Missing required SSL config items

Incomplete SSL configuration.

Complete all required SSL configuration fields.

For GSSAPI, a Kerberos service name is required

Kerberos service name is not configured.

Add a Kerberos service name in the SASL configuration.

For GSSAPI, a Kerberos configuration file path is required

Kerberos configuration file path is missing.

Specify the path to the krb5.conf file.

Unable to retrieve the sig for sig_id

The Situation ID was not found in the database.

Verify that the Situation ID exists and is accessible.

Unable to retrieve the alert for alert_id

The alert ID was not found in the database.

Verify that the alert ID exists and is accessible.

Empty Inform payload with no sig_id or alert_id

The workflow action is missing required parameters.

Ensure that the workflow includes sig_id or alert_id in the details.

Failed to send payload to topic

Kafka connection or topic configuration issues.

Check Kafka connectivity and topic permissions.

Failed to create a new Kafka connection

Invalid connection parameters or network connectivity issues.

Verify the connection configuration and network connectivity.

Log Locations

Workflow Engine Logs:

$MOOGSOFT_HOME/log/moog_farmd.log
In this section: