Big Data: Map Reduce Development

So far we have covered the following topics in the big data. You can click on the hyperlink and go to a specific topic.

- Technology Stack
- hadoop distributed file system (HDFS)
- Map Reduce
- Installing Hadoop ( Single Node)
- Apache Hadoop installing Multi Node
- Big Data: Troubleshooting, Administering and optimizing Hadoop

-Managing HDFS

In this blog, we cover the topic, Map Reduce development. I am not a programmer, but an architect. I cant take you deep into programming as myself dont know it. What i can showing you the programming architecture and thats below.

MAP REDUCE DEVELOPMENT OVERVIEW

Choose IDE

-Eclipse

-Net beans

Configure IDE

-Create project

-Add hadoop JARs

Code it up! ----------> Write Unit Tests--> Compile to jar

                                 Debug Locally             - Run on Cluster

                                - Mockito for unit tests   -View Results and counters

                               -Debug in IDE(On Standalone) -Optimize jobs

                                on subsets of data

- Implement Mapper

- Implement Reducer

- Implement Driver