Kafka

This document explains the steps to configure Kafka as a destination.

Kafka as a destination allows you to consume raw events directly to your Kafka consumer. Lemnisk will stream events coming from different data sources directly to your configured Kafka destination.

Setup Kafka as Destination in Lemnisk

Step 1: Go to Data Pipeline > Destinations.

Step 2: Click on +Create Destination and select Kafka from the destination list.

Image: Create destination

Step 3: Select the source you created and Click Continue

Step 4: Name your destination and click "Submit" Button

Step 5: Navigate to the configuration tab once the destination is created.

Image: Configuring Kafka as destination.

Step 6: Enter the following details to configure your destination.

• Name: Name your destination

• Hostname: Provide the host address where the Kafka is hosted

• Authentication: There are two ways to configure authentication -PLAINTEXT and SASL_PLAINTEXT.

Plaintext configuration

SASL_PLAINTEXT configuration

SASL_PLAINTEXT will further have 3 different authentication mechanisms.

• Topic name: Provide the topic name where Lemnisk will produce the Kafka events.

Step 7: Test your configuration by clicking "Test" Button. If we're able to connect and produce event to the given Kafka config, a success event will be shown otherwise error message will be shown.

Step 8: On successful test, click on "Update" to update the Kafka configuration.

Step 9: Activate the destination.

It may take upto 30 minutes for the destination to work after activating.

Demo: Setup Kafka as destination.