Kafka
Apache Kafka is a distributed event streaming platform that allows you to publish and subscribe to streams of records. Prutan supports publishing events to Kafka topics with various authentication mechanisms.
Creating a Kafka Request
To create a Kafka request,
- Select a Collection, on
hoveror onthree dot click, an icon to create a request will appear. - Click on the icon will open a dialog box.
- Provide label and save to create a request.
- Navigate to the created request and select it.
- Select REALTIME format from the formats
REST, ISO, REALTIME. - Choose KAFKA as the method from the method dropdown.
- Provide the necessary data like
Endpoint, Topic Name, Content Type, Message Body etc.to configure the request. - Navigate to the Authorization tab and configure Kafka authentication (SASL Mechanism, Security Protocol, Username, Password) or None if not authenticated.
- Click Save button to save the request.
- Click Publish button to publish the event to Kafka.
Authentication Configuration
SASL Mechanisms
The following SASL (Simple Authentication and Security Layer) mechanisms are supported:
- PLAIN - Simple username/password authentication
- SCRAM-SHA-256 - Salted Challenge Response Authentication Mechanism with SHA-256
- SCRAM-SHA-512 - Enhanced SCRAM with SHA-512
Security Protocols
- PLAINTEXT - No security (development only)
- SSL - Transport Layer Security
- SASL_PLAINTEXT - SASL over plaintext
- SASL_SSL - SASL over SSL/TLS (recommended for production)
Configuration Fields
Required Fields
| Field | Description | Example |
|---|---|---|
| Method | Select KAFKA | KAFKA |
| Endpoint | Kafka broker address | kafka.example.com:9092 |
| Topic Name | Target Kafka topic | user-events |
| Content Type | Message format | application/json or text/plain |
| Message Body | Content to publish | JSON or text content |
Authentication Fields
| Field | Description | Values |
|---|---|---|
| SASL Mechanism | Authentication mechanism | PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 |
| Security Protocol | Connection security | PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL |
| Username | User identifier for authentication | producer-user |
| Password | Password for authentication | password |
Troubleshooting
| Issue | Possible Cause | Solution |
|---|---|---|
| Kafka connection failed | Invalid broker address or port | Verify Kafka broker URL and accessibility |
| Authentication failed | Invalid credentials or SASL mechanism | Check credentials and verify SASL mechanism |
| Topic not found | Topic doesn't exist or wrong name | Verify topic name exists in Kafka cluster |
| Message not delivered | Network issue or broker unreachable | Check network connectivity and firewall |
Best Practices
- Ensure the Kafka broker is accessible from your network
- Verify the topic exists before publishing
- Use meaningful topic names following your organization's naming convention