Originally Kafka needed ZooKeeper running separately just to manage the cluster — who is the leader, which broker is alive, storing metadata. This meant two systems to manage, extra network calls, and a hard limit on partitions.
From Kafka 3.x, KRaft was introduced as an alternative. From Kafka 4.0 onwards ZooKeeper support is completely removed. Kafka now manages everything internally — brokers vote among themselves using Raft protocol to elect leaders. One less system to run.
Old: Kafka + ZooKeeper (two separate systems)
New: Just Kafka with KRaft (self-managed)
Java — Kafka runs on JVM. Install Java 11 or above.
# Check if Java is installed
java -version
# If not installed
sudo apt update
sudo apt install -y openjdk-17-jdk
# Verify
java -version
# Go to home directory
cd ~
# Download Kafka (or get the link from kafka.apache.org/downloads)
wget <https://downloads.apache.org/kafka/4.0.0/kafka_2.13-4.0.0.tgz>
# Extract
tar -xzf kafka_2.13-4.0.0.tgz
# Go inside
cd kafka_2.13-4.0.0
If you already have the .tgz file downloaded:
tar -xzf kafka_2.13-4.2.0.tgz
cd kafka_2.13-4.2.0
All commands run from inside the kafka directory.
# Step 1 — Generate a unique cluster ID
# This command runs kafka-storage.sh and stores its output in KAFKA_CLUSTER_ID variable
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
# Confirm it got stored
echo $KAFKA_CLUSTER_ID
# Step 2 — Format storage with that cluster ID
# --standalone means single node, no other brokers
# This creates the metadata directory at /tmp/kraft-combined-logs
bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/server.properties
# Step 3 — Start Kafka (keep this terminal open, Kafka runs in foreground)
bin/kafka-server-start.sh config/server.properties
# Step 4 — In a new terminal, verify Kafka is running on port 9092
ss -tlnp | grep 9092
You should see java process listening on 9092. Kafka is ready.