Setting up your Kafka Local Development environment couldn't be simpler. The docker-compose file I explore in this post should get you going straight away.

Docker-Compose

The docker-compose file contains 3 services: ZooKeeper, Kafka itself and Kafdrop.

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
 - zookeeper
    ports:
 - 9092:9092
 - 29092:29092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
  kafdrop:
    image: obsidiandynamics/kafdrop:latest
    depends_on:
 - kafka
    ports:
 - 9000:9000
    environment:
      KAFKA_BROKERCONNECT: kafka:29092

Services

Zookeeper

Zookeeper is available because it is responsible for the coordination and management service of Kafka brokers.

In our dev local instance, a single broker is enough for us to get started.

For more details check: https://zookeeper.apache.org/

Kafdrop

Kafdrop is a lightweight web UI for viewing Kafka brokers, topics, partitions, consumers, and messages.

For more details check: https://github.com/obsidiandynamics/kafdrop

If you need more tools in your docker-compose, Confluent Cloud provides wide varieties to attend to developers' needs, available at https://github.com/confluentinc/cp-all-in-one

Running

Run docker-compose up -d to run the compose detached, and once it is up, go to the browser http://localhost:9000

You should see Kafdrop running:

After execution, run docker-compose down -v to stop and remove all the containers, along with their associated networks, volumes, and images.