Spark SQL and Spark 3 using Scala Hands-On with Labs

A comprehensive course on Spark SQL as well as Data Frame APIs using Scala with complementary lab access

Created by: Durga Viswanatha Raju Gadiraju
CEO at ITVersity and CTO at Analytiqs, Inc

Created by: Madhuri Gadiraju

Created by: Sathvika Dandu

Created by: Pratik Kumar

Created by: Sai Varma

Created by: Phani Bhushan Bozzam
Aspiring Project Manager & Creative UI/UX Designer

Created by: Siva Kalyan Geddada

Created by: Anushka Chakraborty

What Will I Learn?

• All the HDFS Commands that are relevant to validate files and folders in HDFS.
• Enough Scala to work Data Engineering Projects using Scala as Programming Language
• Spark Dataframe APIs to solve the problems using Dataframe style APIs.
• Basic Transformations such as Projection, Filtering, Total as well as Aggregations by Keys using Spark Dataframe APIs
• Inner as well as outer joins using Spark Data Frame APIs
• Ability to use Spark SQL to solve the problems using SQL style syntax.
• Basic Transformations such as Projection, Filtering, Total as well as Aggregations by Keys using Spark SQL
• Inner as well as outer joins using Spark SQL
• Basic DDL to create and manage tables using Spark SQL
• Basic DML or CRUD Operations using Spark SQL
• Create and Manage Partitioned Tables using Spark SQL
• Manipulating Data using Spark SQL Functions
• Advanced Analytical or Windowing Functions to perform aggregations and ranking using Spark SQL

Requirements

• Basic programming skills
• Self support lab (Instructions provided) or ITVersity lab at additional cost for appropriate environment.
• Minimum memory required based on the environment you are using with 64 bit operating system
• 4 GB RAM with access to proper clusters or 16 GB RAM with virtual machines such as Cloudera QuickStart VM

Target audience

• Any IT aspirant/professional willing to learn Data Engineering using Apache Spark
• Python Developers who want to learn Spark using Scala to add additional skill to be a Data Engineer
• Java or Scala Developers to learn Spark using Scala to add Data Engineering Skills to their profile