Azure Databricks & Spark For Data Engineers:Hands-on Project

Real World Project on Formula1 Racing using Azure Databricks, Delta Lake, Unity Catalog, Azure Data Factory [DP203]

Created by: Ramesh Retnasamy
Senior Data Engineer/ Machine Learning Engineer

What Will I Learn?

• You will learn how to build a real world data project using Azure Databricks and Spark Core. This course has been taught using real world data.
• You will acquire professional level data engineering skills in Azure Databricks, Delta Lake, Spark Core, Azure Data Lake Gen2 and Azure Data Factory (ADF)
• You will learn how to create notebooks, dashboards, clusters, cluster pools and jobs in Azure Databricks
• You will learn how to ingest and transform data using PySpark in Azure Databricks
• You will learn how to transform and analyse data using Spark SQL in Azure Databricks
• You will learn about Data Lake architecture and Lakehouse Architecture. Also, you will learn how to implement a Lakehouse architecture using Delta Lake.
• You will learn how to create Azure Data Factory pipelines to execute Databricks notebooks
• You will learn how to create Azure Data Factory triggers to schedule pipelines as well as monitor them.
• You will gain the skills required around Azure Databricks and Data Factory to pass the Azure Data Engineer Associate certification exam DP203
• You will learn how to connect to Azure Databricks from PowerBI to create reports
• You will gain a comprehensive understanding about Unity Catalog and the data governance capabilities offered by Unity Catalog.
• You will learn to implement a data governance solution using Unity Catalog enabled Databricks workspace.

Requirements

• All the code and step-by-step instructions are provided, but the skills below will greatly benefit your journey
• Basic Python programming experience will be required
• Basic SQL knowledge will be required
• Knowledge of cloud fundamentals will be beneficial, but not necessary
• Azure subscription will be required, If you don't have one we will create a free account in the course

Target audience

• University students looking for a career in Data Engineering
• IT developers working on other disciplines trying to move to Data Engineering
• Data Engineers/ Data Warehouse Developers currently working on on-premises technologies, or other cloud platforms such as AWS or GCP who want to learn Azure Data Technologies
• Data Architects looking to gain an understanding about Azure Data Engineering stack