Taming Big Data with MapReduce and Hadoop – Hands On!

Learn MapReduce fast by building over 10 real examples, using Python, MRJob, and Amazon's Elastic MapReduce Service.

Created by: Sundog Education by Frank Kane
Join over 900K students learning ML, AI, AWS, and Data Eng.

Created by: Frank Kane
Ex-Amazon Sr. Engineer and Sr. Manager, CEO Sundog Education

Created by: Sundog Education Team
Sundog Education Team

What Will I Learn?

• Understand how MapReduce can be used to analyze big data sets
• Write your own MapReduce jobs using Python and MRJob
• Run MapReduce jobs on Hadoop clusters using Amazon Elastic MapReduce
• Chain MapReduce jobs together to analyze more complex problems
• Analyze social network data using MapReduce
• Analyze movie ratings data using MapReduce and produce movie recommendations with it.
• Understand other Hadoop-based technologies, including Hive, Pig, and Spark
• Understand what Hadoop is for, and how it works

Requirements

• You'll need a Windows system, and we'll walk you through downloading and installing a Python development environment and the tools you need as part of the course. If you're on Linux and already have a Python development environment in place that you're familiar with, that's OK too. Again, be sure you have at least some programming or scripting experience under your belt. You won't need to be a Python expert to succeed in this course, but you'll need the fundamental concepts of programming in order to pick up what we're doing.

Target audience

• This course is best for students with some prior programming or scripting ability. We will treat you as a beginner when it comes to MapReduce and getting everything set up for writing MapReduce jobs with Python, MRJob, and Amazon's Elastic MapReduce service - but we won't spend a lot of time teaching you how to write code. The focus is on framing data analysis problems as MapReduce problems and running them either locally or on a Hadoop cluster. If you don't know Python, you'll need to be able to pick it up based on the examples we give. If you're new to programming, you'll want to learn a programming or scripting language before taking this course.