Kafka

Apache Kafka is a distributed event streaming platform that allows you to publish and subscribe to streams of records. Prutan supports publishing events to Kafka topics with various authentication mechanisms.

Creating a Kafka Request

To create a Kafka request,

  1. Select a Collection, on hover or on three dot click, an icon to create a request will appear.
  2. Click on the icon will open a dialog box.
  3. Provide label and save to create a request.
  4. Navigate to the created request and select it.
  5. Select REALTIME format from the formats REST, ISO, REALTIME.
  6. Choose KAFKA as the method from the method dropdown.
  7. Provide the necessary data like Endpoint, Topic Name, Content Type, Message Body etc. to configure the request.
  8. Navigate to the Authorization tab and configure Kafka authentication (SASL Mechanism, Security Protocol, Username, Password) or None if not authenticated.
  9. Click Save button to save the request.
  10. Click Publish button to publish the event to Kafka.

Authentication Configuration

SASL Mechanisms

The following SASL (Simple Authentication and Security Layer) mechanisms are supported:

  • PLAIN - Simple username/password authentication
  • SCRAM-SHA-256 - Salted Challenge Response Authentication Mechanism with SHA-256
  • SCRAM-SHA-512 - Enhanced SCRAM with SHA-512

Security Protocols

  • PLAINTEXT - No security (development only)
  • SSL - Transport Layer Security
  • SASL_PLAINTEXT - SASL over plaintext
  • SASL_SSL - SASL over SSL/TLS (recommended for production)

Configuration Fields

Required Fields

FieldDescriptionExample
MethodSelect KAFKAKAFKA
EndpointKafka broker addresskafka.example.com:9092
Topic NameTarget Kafka topicuser-events
Content TypeMessage formatapplication/json or text/plain
Message BodyContent to publishJSON or text content

Authentication Fields

FieldDescriptionValues
SASL MechanismAuthentication mechanismPLAIN, SCRAM-SHA-256, SCRAM-SHA-512
Security ProtocolConnection securityPLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL
UsernameUser identifier for authenticationproducer-user
PasswordPassword for authenticationpassword

Troubleshooting

IssuePossible CauseSolution
Kafka connection failedInvalid broker address or portVerify Kafka broker URL and accessibility
Authentication failedInvalid credentials or SASL mechanismCheck credentials and verify SASL mechanism
Topic not foundTopic doesn't exist or wrong nameVerify topic name exists in Kafka cluster
Message not deliveredNetwork issue or broker unreachableCheck network connectivity and firewall

Best Practices

  • Ensure the Kafka broker is accessible from your network
  • Verify the topic exists before publishing
  • Use meaningful topic names following your organization's naming convention