Ultimate Azure Data Factory: Cloud Data Engineering

Real world Modern Data Warehouse project for Data Engineers using Azure Data Factory, Sql, Data Lake, Databricks [DP203]

Created by: Aravind Suri
Instructor

What Will I Learn?

• You will learn how to build data pipelines in Azure Data Factory (ADF) through a step-by-step approach.
• You will learn how to ingest data in different formats into Azure Data Lake Gen2 using Azure Data Factory (ADF)
• You will learn how to use and build various types of transformations in Azure Data Factory (ADF)
• You will learn hands-on implementations of building generic artifacts in Azure Data Factory (ADF) such as Flowlets and Templates
• You will learn how to transform data into the Medallion layers in Azure Data Lake Gen2 using Data Flows in Azure Data Factory (ADF)
• You will learn how to implement ETL/ELT using Azure Data Factory (ADF) in order to implement a Data Warehouse
• You will learn how to create generic metadata driven pipelines in Azure Data Factory (ADF) to implement the ETL/ELT processes
• You will learn the concepts of the Modern Data Warehouse Architecture and the Delta Lake
• You will learn the concepts of Slowly Changing Dimensions and how to implement them in Azure Data Factory (ADF)
• You will learn how to load transformed data from Azure Data Lake Storage Gen2 to Azure SQL Database using Azure Data Factory (ADF)
• You will learn how to implement a Delta Lake using Databricks Notebook Activity in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2
• You will learn how to transform your raw data into a finished data warehouse using Azure Data Factory (ADF) and then visualize it in PowerBI
• You will learn how to build pipelines using good practices and naming standards as in a typical real-world data engineering project
• You will learn how to implement different types of Triggers in Azure Data Factory (ADF) and how to schedule your data pipelines
• You will learn how to monitor pipelines using Azure Data Factory (ADF), Azure Monitor, and how to recover from pipeline failures
• By the end of this course you will have learnt all the topics required on Azure Data Factory to pass the Azure Data Engineer Associate Certification Exam DP203

Requirements

• Basic understanding of Sql will be beneficial
• Basic understanding of cloud computing will be beneficial
• Experience in Azure is not required, we will learn this step by step within this course as we build the project
• Understanding of data warehouses will be beneficial, but not necessary, we will learn it as we build the project in the course
• An Azure account is required for the course, we will learn how you can create it during the course

Target audience

• Beginners or Students who want to break into the Data Engineering field
• Developers who want to learn Data Engineering
• Data Engineers who want to learn how to implement a Modern Data Warehouse through a step-by-step approach
• Data Engineers/Data Warehouse developers who want to get the skills necessary in implementing cloud based data engineering solutions
• Data Engineers who want to understand how to build and end-to-end solution using Azure Data Factory (ADF)