Kafka for Administrators Training Course

Kafka Administration Essentials

• The role of Kafka in modern data platforms and typical production responsibilities.
• Core operational concepts: brokers, topics, partitions, offsets, and consumer groups.
• Fundamentals of replication: leaders and followers, in-sync replicas, and availability trade-offs.
• Key operational highlights and common terminology found in runbooks.

KRaft Mode and Cluster Design

• KRaft fundamentals: controllers, metadata quorum, elections, and their operational significance.
• Deployment planning: sizing for throughput, partitions, retention, and growth.
• Node roles and layouts: combined versus dedicated controllers, and fault domain considerations.
• Lab: Inspect KRaft metadata, validate quorum health, and interpret controller logs.

Installation, Configuration, and Day-to-Day Operations

• Installation methods (packages, tarball, containers) and best practices for standardization in enterprise environments.
• Core broker configurations affecting reliability: listeners, replication, log directories, and retention.
• Safe service operations: startup order, graceful shutdown, and validation checks.
• Lab: Deploy a multi-node cluster, verify broker registration, and confirm baseline produce and consume operations.

Managing Topics, Partitions, and Data Placement

• Topic lifecycle management via the Kafka CLI: creation, description, configuration updates, and deletion.
• Selecting appropriate partitions and replication factors for real-world workloads, including common anti-patterns.
• Reassignments and balancing: criteria for moving partitions and methods for safely verifying progress.
• Lab: Create topics, trigger partition reassignment, simulate a broker outage, and confirm recovery.

Securing Kafka for Production

• TLS implementation for client and inter-broker traffic: certificates, trust chains, and validation steps.
• Authentication via SASL: selecting common mechanisms and avoiding misconfiguration.
• Authorization using ACLs: least-privilege patterns for administrators, producers, and consumers.
• Lab: Enable TLS and SASL, validate client connectivity, and apply ACLs for application roles.

Observability, Reliability, and Troubleshooting

• Monitoring essentials: controller health, under-replicated partitions, request latency, and disk/network saturation.
• Logs and metrics: analyzing broker logs and exposing metrics via JMX exporter to observability stacks.
• Operational playbooks: rolling restarts, safe configuration changes, and handling disk-full and ISR issues.
• Lab: Build a minimal alert set, diagnose a degraded cluster, and restore healthy replication.

Upgrades and Disaster Recovery Readiness

• Upgrade planning for Kafka: compatibility checks, staging, and rollback strategies.
• Backup and recovery expectations: identifying backup-capable elements, non-backupable items, and configuration recovery basics.
• Overview of cross-cluster replication and the use of MirrorMaker 2 for disaster recovery and migrations.
• Wrap-up: Operational checklist, handover artifacts, and next steps for production rollout.