Course Outline
Module 1: Introduction to the architecture and configuration of the Confluent Apache Kafka cluster
- Kafka's role in modern data pipelines
- Differences between Apache Kafka and Confluent Kafka
- Core components: producers, consumers, brokers, topics, partitions
- Kafka cluster deployment models and scaling considerations
Module 2: Zookeeper Quorum Configuration
- What is Zookeeper?
- Zookeeper’s role in a Kafka cluster
- Zookeeper Quorum size
- Zookeeper configuration
- Implementing SSH on our servers
- Practical: Zookeeper configuration (as a team and as a service)
- Using the Zookeeper Command Line Interface (CLI)
- Practical: Zookeeper Quorum configuration
- Zookeeper internal file system
- Performance factors affecting Zookeeper
- Demonstration of management tools for Zookeeper and Zoonavigator
Module 3: Kafka Cluster Configuration
- Basic Kafka concepts
- Kafka configuration
- Practical: Kafka broker configuration
- Practical: Executing Kafka commands
- Practical: Kafka Multi-Broker Cluster configuration
- Practical: Kafka cluster testing
- Can I connect to my Kafka cluster?
- Advertised.listeners configuration: most important setting
- Topic configuration
- Configuration for downloading and ingesting messages in topics
- Practical: Demonstrating Kafka resilience
- Kafka performance: I/O
- Kafka performance: Network (RED)
- Kafka performance: RAM
- Kafka performance: CPU
- Kafka performance: Operating System (OS)
- Kafka performance: Others
- Practical: How to change Kafka broker configuration
Module 4: Advanced Kafka Configuration
- Landoop Kafka topic user interface, Confluent REST Proxy, Confluent Schema Registry configuration
- Sending and receiving messages (CLI, Java, and Spring framework)
- Monitoring metrics and tools (Confluent Control Center, Elasticsearch, etc.)
- Log files and offset management
- High availability and disaster recovery
- Ensuring high availability through replication
- Tuning producer and consumer performance
- Disaster recovery strategies
- Failover control and data recovery
- Connector configuration
- Kafka Connect implementation
- Kafka security features
Summary and Next Steps
Requirements
- Familiarity with distributed systems and messaging concepts
- Experience with the Linux command line
- Basic understanding of networking and system administration
Audience
- System administrators
- DevOps engineers
- Platform and infrastructure teams
Testimonials (5)
interactive approach of the teacher, not a straight story but acting on the questions from the audience.
Rens - Canon Medical Informatics Europe B.V.
Course - Administration of Kafka Topic
The labs and the slides combine well with Jorge's knowledge and love for Kafka.
Willem - BMW SA
Course - Apache Kafka for Developers
very interactive...
Richard Langford
Course - SMACK Stack for Data Science
Sufficient hands on, trainer is knowledgable
Chris Tan
Course - A Practical Introduction to Stream Processing
Grate skills, examples, very good exercises