Kafka Cluster

A Kafka Cluster is multiple brokers (servers) working together. Instead of one server doing everything, data is spread and copied across multiple brokers. If one broker dies, others already have the data and keep running without any interruption.

Replication

Replication means copying each partition across multiple brokers so if one broker goes down, no data is lost and the system keeps running.

Replication Factor = how many total copies of each partition exist across the cluster.

Replication Factor 1 — No Safety (Dev only)

Broker 1
├── P0 (Leader)
├── P1 (Leader)
└── P2 (Leader)

Only one copy of each partition. Broker goes down means everything is gone. Never use in production.

Replication Factor 2 — Basic Safety

• Each partition exists on 2 brokers — one as Leader, one as Replica
• If Broker 1 dies, Broker 2's replica of P0 automatically becomes the new Leader
• Okay for testing, not recommended for production

Replication Factor 3 — Production Standard

• Every partition is copied across all 3 brokers
• You can lose 2 brokers and the system still survives