Stream processing frameworks for big data: the internals

A deep dive into the internals of Flink, Spark Streaming, Structured Streaming, and Kafka Streams

Created by: Giselle van Dongen
Instructor in distributed stream processing

What Will I Learn?

• The features and internals of Flink, Spark Streaming, Structured Streaming and Kafka Streams.
• How to select the right stream processing framework for a use case.
• The current state-of-the-art of distributed stream processing.
• References to equivalent implementations in all frameworks.
• This is not a programming course! This is a course on understanding how these systems work.

Requirements

• Preferably a notion of distributed systems (e.g. Spark batch API) but not required.

Target audience

• Anybody who needs to get a feeling on how to select the right framework for a use case.
• Anybody who wants to build up firm, in-depth knowledge on the differences and characteristics of these frameworks.
• Anybody who wants to build up a deep understanding of stream processing in general.