Best Hands-on Big Data Practices with PySpark & Spark Tuning

Semi-Structured (JSON), Structured and Unstructured Data Analysis with Spark and Python & Spark Performance Tuning

Created by: Amin Karami
Associate Professor in AI and Data Science

What Will I Learn?

• Understand Apache Spark’s framework, execution and programming model for the development of Big Data Systems
• Learn step-by-step hands-on PySpark practices on structured, unstructured and semi-structured data using RDD, DataFrame and SQL
• Learn how to work with a free Cloud-based and a Desktop computer for Spark setup and configuration
• Build simple to advanced Big Data applications for different types of data (volume, variety, veracity) through real case studies
• Investigate and apply optimization and performance tuning methods to manage data Skewness and prevent Spill
• Investigate and apply Adaptive Query Execution (AQE) to optimize Spark SQL query execution at runtime
• Investigate and be able to explain the lazy evaluations (Narrow vs Wide transformation) and internal working of Spark
• Build and learn Spark SQL applications using JDBC (Java Database Connectivity)

Requirements

• Very basic Python and SQL
• If you are new to Python programming, Don't worry at all, you can learn it freely through my YouTube channel. Subscribe to my YouTube channel and keep learning without any hassle

Target audience

• Beginner/Junior/Senior Data Developers who want to master Spark/PySpark and Spread the knowledge of Big Data Analytics
• If you are new to Python programming, Don't worry at all, you can learn it freely through my YouTube channel. Subscribe to my YouTube channel and keep learning without any hassle