Apache Kafka & Spring WebSocket Complete Reference Guide

What is Apache Kafka?

Apache Kafka is an open-source distributed event streaming platform used for building real-time data pipelines and streaming applications. It allows publishing, storing, and processing large volumes of events in a fault-tolerant and scalable manner.

Kafka is capable of handling trillions of events per day and is widely used for event-driven architectures and real-time analytics.

Kafka Core Concepts

Component Description
Producer Sends messages/events to Kafka topics.
Consumer Reads messages from Kafka topics.
Topic Logical channel where messages are published.
Broker Kafka server responsible for storing and serving data.
Partition Unit of parallelism and scalability within a topic.
ZooKeeper / KRaft Manages metadata, broker coordination and leader election.

Kafka Topic

A Kafka Topic is a logical channel where producers publish messages and consumers subscribe to receive them. Topics can contain multiple partitions for scalability.

Partition in Kafka

A partition is an ordered, immutable sequence of records. Multiple partitions allow Kafka to process messages in parallel.

Kafka Broker

A Kafka Broker is a server responsible for storing messages and serving producer and consumer requests. A Kafka cluster consists of one or more brokers.

Role of ZooKeeper / KRaft

ZooKeeper manages broker metadata, leader election, configuration management and cluster coordination.

Modern Kafka versions are moving towards KRaft mode, which eliminates the ZooKeeper dependency.

Consumer Group

A Consumer Group is a collection of consumers working together to process records from a topic. Kafka guarantees that each partition is consumed by only one consumer within a group.

Consumer Group ID

Kafka Replication

Kafka replicates partitions across multiple brokers for fault tolerance. Each partition has:

If the leader fails, Kafka automatically promotes a follower replica.

What Happens When a Broker Fails?

Kafka Streams

Kafka Streams is a Java client library used to build real-time stream processing applications and microservices.

Kafka Retention Policies

Policy Description
Time-Based Retain messages for a fixed duration (e.g., 7 days).
Size-Based Retain messages until storage limit is reached.
Compaction Retain only the latest record for each key.

Kafka Real-Life Example

Food Delivery Application

Kafka ensures all systems receive the required events without directly communicating with each other.

Common Kafka Use Cases

Real-Time Analytics

Event-Driven Architecture

Data Pipelines

Other Use Cases

Advantages of Kafka

Spring WebSocket

Spring WebSocket provides support for real-time bidirectional communication between client and server using WebSocket protocol.

Common Use Cases

Spring WebSocket Components

Component Description
WebSocket Full-duplex communication protocol.
STOMP Messaging protocol on top of WebSocket.
SockJS Fallback support when WebSocket is unavailable.

Spring Boot Dependency

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-websocket</artifactId>
</dependency>

Spring WebSocket Flow

Client
   |
   v
/app/send
   |
   v
@MessageMapping
   |
   v
/topic/messages
   |
   v
Subscribed Clients

Important Annotations

Annotation Purpose
@EnableWebSocketMessageBroker Enables WebSocket messaging.
@MessageMapping Maps client messages.
@SendTo Broadcasts response to subscribers.
@Controller Handles WebSocket requests.

Kafka vs RabbitMQ

Feature Kafka RabbitMQ
Model Publish-Subscribe Queue-Based
Throughput Very High Moderate
Retention Configurable Deleted After Ack
Ordering Within Partition Limited
Protocol Kafka Protocol AMQP
Best For Event Streaming Task Queues

Kafka vs JMS vs Azure Service Bus

Feature Kafka JMS Azure Service Bus
Messaging Model Event Streaming Queue & Topic Queue & Topic
Scalability Very High Medium High
Cloud Native No No Yes
Throughput Very High Medium High
Retention Configurable Temporary Configurable
Protocol Kafka Protocol JMS AMQP
Best Use Case Streaming Enterprise Messaging Cloud Messaging

Interview Questions

  1. What is Kafka and how does it work?
  2. What is the difference between Topic and Partition?
  3. What is a Consumer Group?
  4. What happens when a Broker fails?
  5. What is Kafka Replication?
  6. What is ISR (In-Sync Replica)?
  7. Difference between Kafka and RabbitMQ?
  8. What is KRaft Mode?
  9. What are Kafka Retention Policies?
  10. What is Spring WebSocket and where is it used?
  11. Difference between WebSocket and REST API?
  12. What is STOMP in Spring WebSocket?

Kafka & Spring WebSocket Interview Quick Reference

1. What is Kafka and How Does it Work?

Apache Kafka is a distributed event-streaming platform used for high-throughput messaging.

Producer --> Topic --> Broker --> Consumer

2. Difference Between Topic and Partition?

Topic Partition
Logical message category Physical subdivision of topic
Stores events Provides scalability & parallelism
Topic: Orders
  +- Partition-0
  +- Partition-1
  +- Partition-2

3. What is a Consumer Group?

Multiple consumers working together to consume messages from a topic.

Topic
  +- Consumer-1
  +- Consumer-2
  +- Consumer-3

4. What Happens When a Broker Fails?

A replica broker becomes leader and continues serving requests.

Broker-1 (Leader) Down
      ?
Broker-2 becomes Leader

5. What is Kafka Replication?

Copies topic partitions across multiple brokers for fault tolerance.

Partition-0
   +- Leader
   +- Replica-1
   +- Replica-2

6. What is ISR (In-Sync Replica)?

Replicas fully synchronized with the leader partition.

ISR = {Leader, Replica-1, Replica-2}

7. Difference Between Kafka and RabbitMQ?

Kafka RabbitMQ
Event Streaming Platform Message Broker
High Throughput Low Latency Messaging
Stores Events Deletes After Processing
Partition Based Queue Based

8. What is KRaft Mode?

KRaft (Kafka Raft Metadata Mode) removes ZooKeeper dependency.

Old:
Kafka + ZooKeeper

New:
Kafka KRaft Mode

9. What are Kafka Retention Policies?

Defines how long Kafka keeps messages.

retention.ms=604800000
(7 Days)

10. What is Spring WebSocket and Where is it Used?

Spring WebSocket enables real-time bidirectional communication.

Used In:

Client ? Server
Real-Time Communication

11. Difference Between WebSocket and REST API?

WebSocket REST API
Persistent Connection Request/Response
Bidirectional Unidirectional
Real-Time Updates Polling Required
Low Latency Higher Overhead

12. What is STOMP in Spring WebSocket?

STOMP (Simple Text Oriented Messaging Protocol) provides messaging semantics over WebSocket.

Client Subscribe:
/topic/messages

Server Send:
@SendTo("/topic/messages")

Quick Interview One-Liners

Question One-Line Answer
Kafka Distributed event-streaming platform for real-time data processing.
Topic Logical channel where messages are stored.
Partition Physical subdivision of a topic for scalability.
Consumer Group Group of consumers sharing message processing workload.
Broker Kafka server responsible for storing and serving data.
Broker Failure Replica automatically becomes leader.
Replication Copies data across brokers for fault tolerance.
ISR Replicas synchronized with leader partition.
Leader Partition Handles all read and write requests.
Follower Replica Copies data from leader partition.
Kafka vs RabbitMQ Kafka is streaming; RabbitMQ is messaging.
KRaft Mode Kafka metadata management without ZooKeeper.
Retention Policy Determines how long Kafka stores messages.
Log Compaction Retains latest value per message key.
Offset Unique identifier of a record within a partition.
Spring WebSocket Framework for real-time bidirectional communication.
WebSocket Persistent full-duplex communication protocol.
REST API Request-response communication over HTTP.
STOMP Messaging protocol used over WebSocket.
@SendTo Sends messages to subscribed WebSocket clients.
/topic Broadcasts messages to multiple subscribers.
/queue Point-to-point messaging destination.
WebSocket Use Cases Chat, notifications, stock updates, gaming.