bgdeveloer-h
.
Introduction

The Motivation for Hadoop

  1. Problems with Traditional Large-Scale Systems
  2. Introducing Hadoop
  3. Hadoopable Problems

Hadoop: Basic Concepts and HDFS

  1. The Hadoop Project and Hadoop Components
  2. The Hadoop Distributed File System

Introduction to MapReduce

  1. MapReduce Overview
  2. Example: WordCount
  3. Mappers
  4. Reducers

Hadoop Clusters and

the Hadoop Ecosystem

  1. Hadoop Cluster Overview
  2. Hadoop Jobs and Tasks
  3. Other Hadoop Ecosystem Components

Writing a MapReduce Program in Java

  1. Basic MapReduce API Concepts
  1. Writing MapReduce Drivers, Mappers, and Reducers in Java
  1. Speeding Up Hadoop Development by Using Eclipse
  1. Differences Between the Old and New MapReduce APIs

Writing a MapReduce Program
Using Streaming

  1. Writing Mappers and Reducers with the Streaming API

Unit Testing MapReduce Programs

  1. Unit Testing
  1. The JUnit and MRUnit Testing Frameworks
  1. Writing Unit Tests with MRUnit
  1. Running Unit Tests

Delving Deeper into the Hadoop API

  • Using the ToolRunner Class
  • Setting Up and Tearing Down Mappers and Reducers
  • Decreasing the Amount of Intermediate Data with Combiners
  • Accessing HDFS Programmatically
  • Using The Distributed Cache
  • Using the Hadoop API’s Library of Mappers, Reducers, and Partitioners

Practical Development Tips
and Techniques

  • Strategies for Debugging MapReduce Code
  • Testing MapReduce Code Locally by Using LocalJobRunner
  • Writing and Viewing Log Files
  • Retrieving Job Information with Counters
  • Reusing Objects
  • Creating Map-Only MapReduce Jobs

Partitioners and Reducers

  • How Partitioners and Reducers Work Together
  • Determining the Optimal Number of Reducers for a Job
  • Writing Customer Partitioners

Data Input and Output

  • Creating Custom Writable and WritableComparable Implementations
  • Saving Binary Data Using SequenceFile and Avro Data Files
  • Issues to Consider When Using File Compression
  • Implementing Custom InputFormats and OutputFormats

Unit Testing MapReduce Programs

  • Unit Testing
  • The JUnit and MRUnit Testing Frameworks
  • Writing Unit Tests with MRUnit
  • Running Unit Tests

Delving Deeper into the Hadoop API

  • Using the ToolRunner Class
  • Setting Up and Tearing Down Mappers and Reducers
  • Decreasing the Amount of Intermediate Data with Combiners
  • Accessing HDFS Programmatically
  • Using The Distributed Cache
  • Using the Hadoop API’s Library of Mappers, Reducers, and Partitioners

Practical Development Tips

and Techniques

  • Strategies for Debugging MapReduce Code
  • Testing MapReduce Code Locally by Using LocalJobRunner
  • Writing and Viewing Log Files
  • Retrieving Job Information with Counters
  • Reusing Objects
  • Creating Map-Only MapReduce Jobs

Partitioners and Reducers

  • How Partitioners and Reducers Work Together
  • Determining the Optimal Number of Reducers for a Job
  • Writing Customer Partitioners

Data Input and Output

  • Creating Custom Writable and WritableComparable Implementations
  • Saving Binary Data Using SequenceFile and Avro Data Files
  • Issues to Consider When Using File Compression
  • Implementing Custom InputFormats and OutputFormats

Common MapReduce Algorithms

  1. Sorting and Searching Large Data Sets
  1. Indexing Data
  1. Computing Term Frequency — Inverse Document Frequency
  1. Calculating Word Co-Occurrence
  1. Performing Secondary Sort

Joining Data Sets in MapReduce Jobs

  1. Writing a Map-Side Join
  1. Writing a Reduce-Side Join

Integrating Hadoop into the Enterprise Workflow

  1. Integrating Hadoop into an Existing Enterprise
  1. Loading Data from an RDBMS into HDFS by Using Sqoop
  1. Managing Real-Time Data Using Flume
  1. Accessing HDFS from Legacy Systems with FuseDFS and HttpFS

An Introduction to Hive, Imapala, and Pig

  1. The Motivation for Hive, Impala, and Pig
  1. Hive Overview
  1. Impala Overview
  1. Pig Overview
  1. Choosing Between Hive, Impala, and Pig

Getting Data Into HDFS

  1. Ingesting Data from External Sources with Flume
  1. Ingesting Data From Relational Databases with Sqoop
  1. Rest Interfaces
  1. Best Practices for Importing Data

Introduction to Scala

scala-s
hadoop-s
hadoop-mr-s
java-s