A rant on design patterns

There are few things a java programmer is expected to demonstrate after a few years of working in trenches and design patterns is cardinal to it. It is not sufficient to implement a design pattern where it is begging to be implemented, to lessen the sagging code logic and improve readability, extensibility or both but also to insert it into every nook, corner and alley of code where it might or might not serve a purpose.
I recently did a code submission for a multinational company and was rejected due to this precise reason - my code was not extensible, they said.

However, what was bitterly amusing for me was the fact that I was shipped with a document that said KISS should be followed and emphasis would be over test driven development. The main point in this frustrating experience is how can one keep it simple and stupid... or minimized without using design patterns. Either I do not have an understanding of the DPs and there is any pattern that can be used to further minimize the code, something that I did in the first place. Merely creating methods and class hierarchy to conform to a pattern(when the implementation is merely 10 lines of code) in every case seems to be an overkill, even if the code is meant to be extensible.

I think as programmer, to create a solution, there are arguably ten ways of performing a task, each with its own advantage and caveat and what is important for a programmer who is reviewing it (for recruitment or as an exercise, while merging open source changes or even for a team member) has to get this basic fundamental correct.

MapReduce2.0: Next Generation Mapreduce using YARN

There has been a lot of long overdue changes in apache hadoop, the most popular framework to perform mapreduce over big data.
As the users of hadoop may recall, one of the vulnerability in mapreduce was the presence of a single node that performed work of job tracker which performed the tasks of job scheduling as well as management and handling errors in map and reduce operations done by these jobs. Until now, there had been workarounds that addressed the failure of this job management node but the separation of work in the jobtracker was not addressed. In particular, several aspects regarding scalability, cluster utilization, change/upgradation, supporting workloads other than MapReduce itself.

YARN(Yet-Another-Resource-Negotiator) separates this problem by splitting up resource management and job scheduler into different daemons that are then applicable on applications - either as a single job or a graph of interrelated jobs. Resource management is done by a global ResourceManager(RM) that resembles the old job tracker, but exists independently of the job or its application. To perform the jobs for each application, ApplicationMaster(AM) is used which acts as an intermediary between the RM and the nodes, which are in turn manipulated through NodeManager(NM) and it relays the node health and resource requirements to the RM as needed.

The YARN archiecture

A key advantage of YARN is that it supports algorithms other than MapReduce. There are problems that need to have intermediate or realtime results while processing and traditionally can't be used in mapreduce such as graph processing. Simply put in context of hadoop, if one node's results are required by another after or during processing of one or the both records, then batch processing is the answer.

People generally get confused in the context of the newer version of hadoop(the top level project) with MRv2 as the hadoop supports both the MR1 as well as MRv2. The change is simply in the architecture, that can determine the performance(especially post the map stage - wherein the mapreduce implementations often left nodes underutilized at the reduce phase) of the application in use. One key thing to note here is the fact that the YARN is a framework in itself and what hadoop does is to run mapreduce as a job on the YARN. Thus hadoop preserves its existing API but reaps the benefits arising out of dynamic node allocation done internally by YARN.

Musings on refactoring

I happened to be across the refactoring in ruby book, which is the
adoption of the refactorings book by fowler from java into ruby. On
reading, one thing which strikes you is the fact that refactoring is persistent across languages and platforms. Java is verbose and allows various coding styles and idiosyncrasies which is evident why lot of java programmers churn
out code that is unnecessarily complex. The proliferation of IDEs has also encouraged this malpractice of creating not just sub-optimal but also unnecessarily complex code.

However, finding the same in ruby has traditionally been difficult as ruby, like its japanese inventor has been neat and concise in its apporach, allowing basically for a lot of bang for a small amount of code. However, as the community grows, there is an acute need to maintain the standards in ruby as well as the developer base is exploding exponentially in ruby. In my opinion, refactoring is a practice that every developer needs to apply and enforce amongst their team as a cardinal rule in programming. Applying this context in the updated refactorings book, I feel while using ruby the answers have become more to the point, but at the cost of grossly sloppy code. It could have better demonstrated in a language that is equally neat and offers variety like java. Python obviously springs into mind where at one place there is a C like procedural code and at the other end there are python experts proclaiming the use of pythonic code.

Having the same refactoring into another language feels alien for a while, but then it is a bliss to realize that same problem existed into another language as well, it was just that a learner like me got carried away in the syntactic differences.

At the end of the day, it is the learner who needs to decide what to and more importantly what not to include in the changes to be made in the codebase.

Pretty Date & Time representations in Java

Date and time representations in java has never been easy as both the Date as well as Calendar APIs have been poorly designed. In contrast, ActiveSupport gem in ruby allows for time to be declared constructively. For example, the time can either have arithmetic operations like 45.minutes + 2.months or can call constructive operations such as 1.year.from_now .
In java, this is not the case but thanks to PrettyTime library, this is possible from java as well. This is an easy-to use library which can be used easily without much of a fuss.
Apart from simple time substitutions, Pretty time can also pick up values within the String.

Consider the following example:
"I did this work three days ago"
"and I did that one week ago"

when processed by the following code,

List parse = new PrettyTimeParser().parse("I did this work three days ago");
parse.addAll(new PrettyTimeParser().parse("and I did that one week ago"));
for(Date dt : parse)
System.out.println("date obtained: "+dt);
results in :
date obtained: Tue May 28 18:46:00 IST 2013
date obtained: Fri May 24 18:46:00 IST 2013

There are various use cases that are still to be covered and working on them looks promising. I am giving this a try on a fork of the same project. I am hoping for some constructive feedback and a productive outcome from this exercise.

Book Review: Learning Play Framework!

This is the book review of learning play framework2 by Andy Petrella.
The book focuses on scala based development on play framework. While the
book focuses on existing users of play, it gives a fast paced
introduction to scala and the level of complexity of the book is
slightly above the developer documentation, but not all the way high up
for serious deployment. There is a chapter for the uninitiated in scala,
which I feel is redundant as a section alone will do, given the
proliferation scala has made in java community. Also, it is easy to get
lost while following the samples, so I'd recommend that you keep the
sample questions handy while using this book.

The book starts with setup of play and a bit about its ecosystem, which is redundant. Next, it gives a fast-paced review of scala(it is not complete and I'd recommend the freely available 'scala for the impatient' book as a learning point). The book then demonstrates scala based templating mechanism in play and also plays with css configuration using LESS. It then delves into data management where data related views and routes are explained. This chapter also provides with the changes made in scala 2.1.1 and provides information about ebeans. The next couple of chapters  cover some of the exciting new features that were not present earlier and it deals with multipart content types as well as realtime client interactions through websocket.
A chat application is demonstrated and usage of Akka is shown to create non blocking multi-threaded access for clients, which is quite neat.

Interacting with web services is dealt next with a twitter integration into an existing web application is displayed. and the book caps off with chapters on testing and deployment. While testing is a chapter that I feel should have come before to further bring the TDD into development, the deployment chapter is totally updated. I was expecting to see the likes of jenkins and metrics tool in CI deployment, but there was various cloud vendors and their offerings that were waiting to be demonstrated.
Overall, this book is a great book to start with Play Framework2 and also does not disappoints with the ecosystem surrounding play.
Disclaimer: I received a copy of this book through Packt Publishing.
You can check out more about the book via http://www.packtpub.com/learning-play-framework-2/book