Dev Zone

In this post, We are gonna see secondary sorting with Avro file format, using AvroJob Api and its gonna be two part series, MapReduce V1 Api (org.apache.hadoop.mapred) MapReduce V2 Api (org.apache.hadoop.mapreduce) In this post we are going to see example in MapReduce V1 Api. So lets start, Here's the Avro  Input Schema that we are going to use in this example. Sample Data which we are going to read as input file, for viewing purpose data is shown in JSON format. but it was read as  avro format in mapreduce program. Secondary sorting means the values sent to the Reducer should be sorted based on some criteria. you can refer more about secondary sorting here . In this example, Reducer Iterator values should be sorted based the TimeStamp of above data, but the Reducer Input Key should be grouped based on Id of above sample data. We need use MapOutput Key Schema for Secondary Sorting(Natural Key + Composite Key).Here Natural Key is Id and Composite Ke...

         Builder design patter is used to create an object with optional instance attributes.If you want some introduction about design patterns please refer here.         So, first step is to know what wikipedia says about builder design pattern, The intent of the Builder design pattern is to separate the construction of a complex object from its representation.         Second Step is to find an real world example (behavior) of builder design pattern. Builder pattern is similar to Buffet system, where diners will serve themselves (they pick what dishes they want) , here we need map each dish as instance attributes and people as the client class need to create a object with optional attributes.                 Now the fun part, below codes shows the implementation of builder pattern         Here's th...

       Keeping git repositories in sync is usually and easily done using git pull and git push . However, for those rare times when you don't have network connectivity (or limited connectivity) and still needs to transfer the contents of a repository to somewhere else, there's git bundle .       Another use case, would be like you have hired external contractors to build some application, now you need to get code from them to your organization internal private Git Server hosted and maintained by your organization  and it doesn't exposed to public. So instead of sending .zip format of code, they can send git bundle. What Git Bundle Covers?       Git bundle file is a full repository in a single file, you can have branches , commit history , tags , basically everything you expect in a repository, but it's all contained in a single file.This makes sharing the repository or moving the full repository pretty ...

      We have various ways to create singleton class in java, first of all what is singleton and why it is required ?       Singleton class will restrict you by creating a new object using new keyword, It provides the same single instance for the single JVM instance. If you want some introduction about design patterns please have a look it my post here .This post is little bit too long, so please hold on  your breath.       So the first step is to know what wikipedia says,       The singleton pattern is a software design pattern that restricts the instantiation of a class to one.       Second step is to map singleton pattern to the real world example,             Each country will have a single President   post, for every some years (normally 5) the people for the post will change but the country will hold only one Preside...

    If you came here means you want to know how to learn it(as the topic says) and not to what and why and all those , anyhow I will cover it through from what it is to the how to , so here's the definition,  Design patterns are solutions to the repeatable problems occurring in the software design. why we need Design Pattern?    When we program, we write description intended for two audiences, compiler and the other developers , sometimes it may help yourself after we look at the code written by last year or sometimes even within months. Compiler will be happy until it satisfies the language specification. Developers need to know what functionality it is and why this design and also they want to know what is Robust and Fragile.      Documentation will provide those things, but it changes once the source code modification happens, Design patterns will provide a effective communication between developers they share common vocabularies ...