Towards RESTful Web Services

When we talk about distributed applications, internet is the enabling technology that comes to the mind. Sure, one can think of many other ways and protocols of performing a software feat on more than one computer at the same time, but the fact that internet or the World WideWeb is the biggest network out there remains true and has the appeal of its ubiquity in the implementation of any such distributed application.
You can even think of a website as a distributed application, but here, the client (or you) is a passive person/computer that can only use what is fed to him. Even dynamic websites ultimately generate static content (or for that matter, even Web 2.0, which is built on user experience rather than anything else), which even if it appears as a distributed application, there is no such distribution of computation going on. However, in a truly distributed application, the portions of application reside in different computers and communicate with each other via internet. This interchange of the necessary data and process requests is what is needed in these kinds of applications.
Before I further befuddle you from the idiosyncrasy of complex distributed application technologies that are widely used like WS-* stack, we need to understand the simple phenomenon that on the web, the content generated on the web pages is in form of HTML, that is read by the web browsers on your computers and displayed to you as a web page. However, if we change this format to a different one (still textual, not changing into binary ) like XML or JSON, the end users of our application become different.
So ultimately, there is not much difference between a web site and a web service as the former can be used interchangeably with the latter and what make a web site easier to be used by a human being can be applied to a web service, which can be applied over internet on the same manner, to a program. The common web service protocol has been SOAP, which is essentially an application layer on top of whatever internet it is operating on. This created a lighter way of creating distributed and interoperable applications, but at an increasing cost of complexity. Instead of reinventing the wheel, wouldn't it be simply great if we simply set forth the best of the internet in a mix and obtain something refreshingly easy, elegant and say useful for our purposes.
This is what REST or REpresentational State Transfer is all about. It is not just an alternative method of creating web services, but is arguably the easier way of creating them as it addresses the addressing and discovery concerns based on the established design patterns of internet. The simplicity of this approach is not its weakness as you probably might be thinking, but its strength. Other web service provider technologies can claim maturity and tools to hide complexity, but the changing face of REST is negating this limitation of itself in this regard. In my future posts, I'll explain how the changing face of REST is going to make it a force to reckon with in future.

Scalable Application Architecture with Memory Caches

This post is dedicated towards an important feature that is required in creation of highly performant and scalable applications. Data has traditionally been stored in specialized software, aptly known as database. However, given the massive use of some specific data which is generally static in nature in applications that permit large number of users(thousands, if not more) to interact with the application simultaneously. Using a memory storage instead of a hard disk file results in huge performance gain as the requests pertaining to that data would not be read from a database located on a disk.Think of a in-memory data cache as a robust, persistent storage that can be used in certain transient tasks where the data is stored in RAM of cache machines.

The caching mechanism (framework responsible for maintaining the cache) has to do the following main tasks :

• Maintain a cache (Obvious one)
• Determine the pattern of requests (which requests are more in number)
• Flush out resources and load new ones (different strategies can be used here, most common one is LRU)
• Make sure that the data maintained in cache is correct (and if not, then to which extent)
• Enforce consistency of cache across different machines (Generally this design is needed in cloud or clustered machines)
• Maintain resource utilization (Evict the data from cache if server need increases)

Caching is generally found in distributed applications which are targeted to be used by large amount of people if they are not already in production environment. Today, most of the memory cache frameworks do not offer a synchronization mechanism between the data stored in the database and the memory cache. To overcome this problem, we need to explicitly set an expiration value of the cached object so that it gets refreshed upon requests after a certain period. This performance optimization can be done not only for database specific operations, but also on other data such as repeated web service calls, computation results, static content, etc.

A popular interface standard for java is the JCache, which was proposed as JSR 107. A memory cache software, eg: memcatched [http://www.memcached.org], stores key value pair of data. As soon as a request is generated, the key values are searched, which results in a cache hit or a cache miss scenario. JSR 107 has been adopted in different implementations, one of which is the Google App Engine, which is a cloud platform supporting python and java runtimes. Here, this comes in form of a memcache service for the java runtime. This can be better explained with the following example :


import com.google.appengine.api.memcache.MemcacheService;
import com.google.appengine.api.memcache.MemcacheServiceFactory;
......

MemcacheService cache=MemcacheServiceFactory.getMemcacheService();

.....

cache.put("key","value");

....

object=cache.get("key");

.....
cache.delete("key");

This really makes application scalability easier for the developers (Cloud environments do induce the responsibility of creating applications that can scale quickly). As of now, similar feature doesnt exist in Windows Azure, but what the future holds for this technology cannot be speculated.
Thus it is not surprising that this technology is used in prime websites like YouTube, Wikipedia, Amazon,SourceForge, Metacafe, Facebook, Twitter, etc and that too in large quantites (for eg: Facebook uses over 25 TB of memcache). So, it is imperative for software developers to understand the working and development of this technology.

My professional life as of now

Like the other years, this year too has been a hectic one for me so far and building upon my existing skills, I've tried to contuniously improve myself as a human being and as a professional. My policy so far has been 'Kaizen', which is the Japanese word that literally means continuous improvement.
So far, the greatest improvement that I've made in the first quarter of the current year has not on the programming languages or frameworks, or for that matter, any software development practice, but on allied fields. Currently, I am working upon solving business problems via alternative means like ERP, CRM and CMS softwares practically.
As I write this blog post, alternatively, I am reading an excellent e-book titled 'Cases on Strategic Information System' that has been given to my entire class of Computer Applications by our Management Information Systems teacher. This contains 24 insightful case studies on the subject and although I have gone through a handful of them, they are really ponderable problems.
Apart from this management hogwash, rest is business as usual. At the core of my heart, I still have misgivings about management as an ethical study and am satisfied that I resisted the temptation to take a managerial course after doing my bachelors.
At my college, the brand value of Sumit Bisht (yes, that's me) continues to rise. During the past month, I have successfully conducted a technical hands-on lab and gave various presentations in front of the entire department. Apart from this, I have frustratingly and agonizingly tried to learn programming for business intelligence and reporting and am making some headway there too. Hopefully, this strategic initiative yields me rich returns in future.
That said, the software developer in me has not taken a sabbathical and thanks to the Google Summer Of Code 2010, I am actively involved in open source software development and even if my proposal is rejected, I would carry on in these exciting projects. This is the first time for me in this event and quite frankly, I am impressed with the amount of attention this activity has generated. I checked into a lot of projects and finally selected a couple of projects which were doable in my opinion with ease. Thus, this summer is going to be as promising and engaging as the last one.
Ciao for now

Programming Wisdom

Recently, I had the opportunity to read the book, '97 things every programmer should know' by Oreilly publications. Normally I do not read computer books from cover to cover, but this was different. The book had 97 two-page insights by some very experienced programmers and people who really knew the stuff. As I read each pearl of wisdom, it was an engaging experience as every page seemed to yield me with a new insight.

However, as there are some common best practices in our trade, there was some redundancy in the text as well. Nevertheless, it was a journey that is really appreciated like us newbies (without any actual experience). So, like an engaging novel, this book really is a treasure trove for best practitioners and average programmers alike.

This was similar to my previous experiences while reading expert advice. Last year, I had the opportunity to read Bruce Tate's 'Beyond Java'. However, this text differs from Tate's as it is more 'politically correct' and does'nt focusses on advices by a single person. What is also wonderful about this book is that it is platform and language independent and the advices that it offers regarding programming are certainly most talked about in the industry.

As a conclusion, I'd like to give a tribute to the authors as they were simply too wonderful in their explanations(rather than engaging in an obscure jargon) Hats off to them.

Here's the listing of the wisdom(drum-roll) given in this book

1.       Act with Prudence

2.       Apply Functional Programming Principles

3.       Ask, “What Would the User Do?” (You Are Not the User)

4.       Automate Your Coding Standard

5.       Beauty Is in Simplicity

6.       Before You Refactor

7.       Beware the Share

8.       The Boy Scout Rule

9.       Check Your Code First Before Looking to Blame Others

10.   Choose Your Tools with

11.   Code in the Language of the Domain

12.   Code Is Design

13.   Code Layout Matters

14.   Code Reviews

15.   Coding with Reason

16.   A Comment on Comments

17.   Comment Only What the Code Cannot Say

18.   Continuous Learning

19.   Convenience Is Not an -ility

20.   Deploy Early and Often

21.   Distinguish Business Exceptions from Technical

22.   Do Lots of Deliberate Practice

23.   Domain-Specific Languages

24.   Don’t Be Afraid to Break Things

25.   Don’t Be Cute with Your Test Data

26.   Don’t Ignore That Error!

27.   Don’t Just Learn the Language, Understand Its Culture

28.   Don’t Nail Your Program into the Upright Position

29.   Don’t Rely on “Magic Happens Here”

30.   Don’t Repeat Yourself

31.   Don’t Touch That Code!

32.   Encapsulate Behavior, Not Just State

33.   Floating-Point Numbers Aren’t Real

34.   Fulfill Your Ambitions with Open Source

35.   The Golden Rule of API Design

36.   The Guru Myth

37.   Hard Work Does Not Pay Off

38.   How to Use a Bug Tracker

39.   Improve Code by Removing It

40.   Install Me

41.   Interprocess Communication Affects Application Response Time

42.   Keep the Build Clean

43.   Know How to Use Command-Line Tools

44.   Know Well More Than Two Programming Languages

45.   Know Your IDE

46.   Know Your Limits

47.   Know Your Next Commit

48.   Large, Interconnected Data Belongs to a Database

49.   Learn Foreign Languages

50.   Learn to Estimate

51.   Learn to Say, “Hello, World”

52.   Let Your Project Speak for Itself

53.   The Linker Is Not a Magical Program

54.   The Longevity of Interim Solutions

55.   Make Interfaces Easy to Use Correctly and Hard to Use Incorrectly

56.   Make the Invisible More Visible

57.   Message Passing Leads to Better Scalability in Parallel Systems

58.   A Message to the Future

59.   Missing Opportunities for Polymorphism

60.   News of the Weird: Testers Are Your Friends

61.   One Binary

62.   Only the Code Tells the Truth

63.   Own (and Refactor) the Build

64.   Pair Program and Feel the Flow

65.   Prefer Domain-Specific Types to Primitive Types

66.   Prevent Errors

67.   The Professional Programmer

68.   Put Everything Under Version Control

69.   Put the Mouse Down and Step Away from the Keyboard

70.   Read Code

71.   Read the Humanities

72.   Reinvent the Wheel Often

73.   Resist the Temptation of the Singleton Pattern

74.   The Road to Performance Is Littered with Dirty Code Bombs

75.   Simplicity Comes from Reduction

76.   The Single Responsibility Principle

77.   Start from Yes

78.   Step Back and Automate, Automate, Automate

79.   Take Advantage of Code Analysis Tools

80.   Test for Required Behavior, Not Incidental Behavior

81.   Test Precisely and Concretely

82.   Test While You Sleep (and over Weekends)

83.   Testing Is the Engineering Rigor of Software Development

84.   Thinking in States

85.   Two Heads Are Often Better Than One

86.   Two Wrongs Can Make a Right (and Are Difficult to Fix)

87.   Ubuntu Coding for Your Friends

88.   The Unix Tools Are Your Friends

89.   Use the Right Algorithm and Data Structure

90.   Verbose Logging Will Disturb Your Sleep

91.   WET Dilutes Performance Bottlenecks

92.   When Programmers and Testers Collaborate

93.   Write Code As If You Had to Support It for the Rest of Your Life

94.   Write Small Functions Using Examples

95.   Write Tests for People

96.   You Gotta Care About the Code

97.   Your Customers Do Not Mean What They Say

It’s not that great Idea, Sirji!

This post is regarding the false nature of so called green IT revolution created by the media. To a large extent, adoption of technology has helped us save the pollution of earth, but are e-devices really giving us benefit that is promised against the regular ways. This Interview with Don Carli Executive Vice President of SustainCommWorld LLC, and Senior Research Fellow with the Institute for Sustainable Communication addresses some of the false claims made by e-reader companies.

Although the penetration of such devices is not much in our country, it is just a matter of time before we see everyone preferring e-readers instead of books. Apart from e-readers, a large number of various smart devices today promise a cleaner environment, but if recycling them is the only way to achieve a true green way, then why cannot we improve the recycling of paper in the first place?

In the article mentioned above, one of the disturbing things was the increase in the energy requirements of data centers in an exponential manner. Again here too, various companies are quick to highlight their green data centers that reduce their energy requirements on non-renewable energy sources. But one wonders if there are these green data centers, then why is the energy requirement of these data centers still increasing.

Hence it is of importance to the media that they portray the right state of so called environment friendly products, and not false claims like a leading telecom provider of India that encourages mobile usage to save trees in its newly launched promotion campaign.