Apache Airflow on AWS EKS: The Hands-On Guide

Learn to author, schedule and monitor data pipelines through practical examples using Apache Airflow

Created by: Marc Lamberti
Apache Airflow Expert, Data Engineer

What Will I Learn?

• Create plugins to add functionalities to Apache Airflow.
• Using Docker with Airflow and different executors
• Master core functionalities such as DAGs, Operators, Tasks, Workflows, etc
• Understand and apply advanced concepts of Apache Airflow such as XCOMs, Branching and SubDAGs.
• The difference between Sequential, Local and Celery Executors, how do they work and how can you use them.
• Use Apache Airflow in a Big Data ecosystem with Hive, PostgreSQL, Elasticsearch etc.
• Install and configure Apache Airflow
• Think, answer and implement solutions using Airflow to real data processing problems

Requirements

• At least 8 gigabytes of memory
• Some prior programming or scripting experience. Python experience will help you a lot but since it's a very easy language to learn, it shouldn't be too difficult if you are not familiar with.

Target audience

• People being curious about data engineering.
• People who want to learn basic and advanced concepts about Apache Airflow.
• People who like hands-on approach.