Course Outline
Course Outline: Day 1
• Introduction to the core concepts of data streaming
• Foundational differences between batch and real-time processing
• Basics of event-driven architecture
• Typical industry applications and use cases
• An overview of the streaming technology ecosystem
Day 2
• Design patterns for streaming architectures
• Fundamentals of distributed messaging systems
• Roles of producers and consumers
• Understanding topics, partitions, and data flow mechanics
• Strategies for data ingestion
Day 3
• Concepts and frameworks for stream processing
• Contrasting event time with processing time
• Windowing techniques and their practical applications
• Stateful stream processing mechanisms
• Basics of fault tolerance and checkpointing
Day 4
• Data transformation within streaming pipelines
• Applying ETL and ELT methods in real-time systems
• Schema management and evolution strategies
• Stream joins and data enrichment
• Introduction to cloud-based streaming services
Day 5
• Monitoring and observability practices in streaming systems
• Essentials of security and access control
• Performance tuning and optimization techniques
• Comprehensive review of end-to-end pipeline design
• Real-world case studies, including fraud detection and IoT processing
Testimonials (2)
A journey through the Spark world: a very intense course. DSL, spark sql, partitioning vs bucketing for me.
Georgiana Elisabeta
Course - Apache Spark Fundamentals
Hands on exercises. Class should have been 5 days, but the 3 days helped to clear up a lot of questions that I had from working with NiFi already
