The curious case of multithreading Sqlite

In a recent project, I've embarked upon the process of saving information in a quick and lightweight manner for which the serverless database, sqlite was ideal. This is not only a flexible database supporting dynamic data types, but also an extendible one used from embedded web browsers and mobile platforms to rails based websites(good riddiance, mysql) .The database is so lightweight that you'll find yourself exclaiming it as db. It is less than 100 kb in some cases and can be used as a memory resident database for the performance conscious folks and applications.
However, the support of transactions and scalability leave a lot to desire as the documentation cleary states the goals of this database is not these.
In my case, getting started was just the matter of finding the jdbc driver and it s documentation from its website. However, upon customization, there was a need for normalization which in turn required a sequential read/write operations. As soon as this was done, the multi-threaded application that was accessing started experiencing concurrency issues around the database.
This was resolved via using a little used jni wrapper that I discovered, sqlite4java. This not only solved the concurrency issues, but also enabled transactions. The beauty of this approach is also the fact that it uses a single connection as opposed to a pooled/fresh connection per every request, which tends to be a real performance bottleneck as we scale up our application.

For such a nice framework, it is a surprise that these changes have not permeated to the java/native jdbc drivers so far and the users have to cope up with its api at the moment. This could be probably because of its tight integration with the underlying C-based api that is quite performant as compared to its
alternatives in python and java, effectively making it non compliant with the jdbc specification.

As the single connection served entire applications, exposing it as a singleton was the easiest approach :


static SQLiteQueue queue = new SQLiteQueue(new File("BiddingsDB"));

static {
queue.start();
}

For providing a transaction-like support, there was a need to wrap the existing operations in this queue :

public static void databaseInsert(final List

PC Gaming on Linux

Many people consider Linux as a serious OS for computing, but there is a fun aspect to it that is not really known as there is no marketing done for these games. However, for a moderate gamer, linux platform is satisfying especially if you are looking for playing games for your amusement without going too heavy on your wallet. Linux has also improved on the UI front considerably as now we can have some seriously GPU intensive graphics through out of box programs like compiz.

Cool 3D Desktop with Compiz

Those looking for a quick solution can use http://live.linux-gamers.net that allows you to create and start gaming from the live CD/USB directly.

In addition to the point and click games(like card based games),  you can have some serious fun with 3D games that PC gamers have become familiar with. Be it a fast paced action game like AssaultCube or a racing game like TuxKart, there are options for everyone.

TuxKart  : Funny racing game

AssaultCube : Fast paced action, a la Quake3 style

Thanks to ID software's initiative of providing a linux binary(and their stand on DirectX framework), we can install and play a game like DOOM3 on Linux http://sites.google.com/site/lgmscripts/scripts/games/doom-3 (which is definitively on my TODO list:)

There are certain caveats to this excercise too. Since my laptop has a Nvidia GeForce 8200MG card, I had to use a non open source binary driver as the cannonical one was not working well on Ubuntu. If you consider the freedom and the non presence of any cost/license, it forms a good overall personal entertainment package.

Domain-Driven Design Using Naked Objects

I just had a chance to read a newly released book, 'Domain-Driven Design Using Naked Objects' by Dan Haywood [http://www.pragprog.com/titles/dhnako] that provides an insight into the world of DDD. If nothing else, this book is for techies & management people alike. Although Naked Objects is covered as the implementation framework (to explain practically, all aspects of DDD), this book serves as an excellent introduction text for all of us who are new into this concept of crafting an enterprise application according to domain. This book also deserves to be read specially because there has been considerable buzz around this 'Domain' stuff in the recent past.

According to the book ,
'Domain-driven design is an approach to building application software that focuses on the bit that matters in enterprise applications: the core business domain. Rather than putting all your effort into technical concerns, you work to identify the key concepts that you want the application to handle, you try to figure out how they relate, and you experiment with allocating responsibilities (functionality). Those concepts might be easy to see (Customers, Products, Orders, and so on), but often there are more subtle ones (Payable, ShippingRecipient, and RepeatingOrder) that won’t get spotted the first time around. So, you can use a team that consists of business domain experts and developers, and you work to make sure that each understands the other by using the common ground of the domain itself.'

Since being a domain expert is the next step for any developer naturally, as his experience increases, there is need to have more insight into the business stuff, instead of just the software. UML although does a great job for explaining the Object Oriented stuff, but ponder for a second, how is it really going to help a businessman that is interested in getting more profits. Naked Objects framework (http://www.nakedobjects.org) is based on the design pattern with same name(http://en.wikipedia.org/wiki/naked_objects) is an open source framework (only for java, .NET is a commercial one) that converts simple bean/components automatically into an interface(read as  multiple applications).
Don't yet confuse this with prototyping because DDD incorporates both the developer and the domain expert teams and we are not just creating the UI.
DDD's 2 central premises explained :

• A ubiquitous language for integrating & easing communication between domain experts instead of 2, which is the existing norm(like code & UML)
• Using model-driven design that aims to capture the model of the business process. This is done in code, rather than just visually, as was the case earlier.

Naked Objects
The java based Naked Objects(NO) framework is an evolutionary step forward from Rails (and its other avatars : Grails, Spring Roo, asp.net MVC, etc) that focuses more on M & V rather than MVC and provides much more domain-specific applications in turn resulting in flexibility for all.
A typical NO application consists of multiple sub-projects like the core domain, fixture, service, command line and webapp project through a maven archetype. The coolest thing  is that NO automatically displays the domain objects in an O-O based UI that offers display in more flexible manner than any other IDE.
NO also challenges the common frontend-middleware-backend convention and instead applies the Hexagonal architecture (http://alistair.cockburn.us/Hexagonal+architecture) that deals with the bigger picture in mind. The development in this framework is pojo centric and is heavily based on annotations, which should be pretty much regular stuff for any JEE developer. Also, during my initial evaluation of the framework, the code that's being generated during the development is of maintainable quality, which is virtually essential for maintenance and scaling in any enterprise application.

Hence this book, and its field of study is highly recommended for any enterprise developer/ team/ manager/ domain expert and as is repeatedly mentioned, becomes highly important when one has more years of experience under his belt. I am continuing my exploration in this and if it is really useful for me, would post some exercises here.

10 Years of Ruby!

Let me recollect not very recent past. It was 2008 and recession was at its peak. Back then, I was largely a self-taught developer and had a working knowledge in struts & ejb (the java heavyweights at that time). After unsuccessfully trying to yield a job, I finally decided to pursue my masters in computer applications (out of necessity, or so it seemed back then).
During this frustrating period, I heard about Ruby On Rails, which was touted as the next-gen platform/language. As with other people, I was quite scared of it as the name sounded very unfamiliar. So, I didn't pursue it. However in the summers of 2009, as I was idling across the netbeans.org looking for some interesting activity to do, I came across this '5-minute Weblog' exercise on rails and decided to have a go.
Believe me, it was one of the turning point in my career and I spent more than an hour figuring out exactly how I was able to generate a web application just like that using practically no big tools.


As the things turned out, I was not the only one who leaded this entirely new approach. During a recent magazine interview(http://www.pragprog.com/magazines/2010-12/chad-fowler-on-ruby), Chad Fowler, a prominent figure on Ruby Conference and a ruby authority echoed similar views.
During this interview, it was interesting to observe the maturing of ruby as a language and a platform (performance wise and lightweight for mobile devices).

My current job deals with java, however Ruby & Scala have kept my other side of programming (hacking & open-sourcing) buzzing with activity and with a healthy mix of curiosity and satisfaction.

Increasing usage of Functional Programming in driving scalable architecture

One of the most exciting technologies that I've been pursuing lately is nothing new, but as an old wine in a new bottle, provides an alternative method to solve emerging issues for addressing performance. I am referring to Functional Programming, that has been growing in importance during some period in past few years due to various ongoing development in today's huge data processing applications used in enterprise environments. I've been seeing the rise of this otherwise academic language, and is increasingly used/considered to be usable for the past few years due to various ongoing developments.

Features
Functional programming provides various features that separates it as a different paradigm for programming( http://en.wikipedia.org/wiki/Functional_programming ). I'll be explaining various features as per this post's context.
Higher Order Functions : FP revolves around functions! Think of this as a small version of class in OOP. If you've worked with, say anonymous or inner classes in the past. You've unknowingly been doing FP in OOP, and needless to say, it was difficult to learn and maintain. Generics solve the same problem as First Class Functions, but the resultant implementation leaves a lot to be desired.
Recursion : Again a handy feature that provides a lot of bang for small amount of buck, or code! If you know how to use your functions, you can do a lot with your code without having to write boilerplate or plumbing code in your algorithmic implementations.
Immutability : Now this is what I am talking about! We all are familiar with multi-threaded programming, yet the tools for using it are hardly ever used as multi threaded programming involves sharing of resources, which is not a good thing. In all other programming paradigms, we are familiar with the foll construct :
      int a = 0;   Initialization at this line in the memory

   a = 1;       Memory address pointed by a gets 'Updated'

The problem lies with the second statement as we don't know when and more importantly which thread is going to invoke it. But if you recall from your Mathematics, we used to have following notations there:

   let a=0      Initialize a

   a=5          Assign a something new, and ignore previous state

As the value of a is immutable, we can safely have as many threads to it as we need. This means, as our applications are thread safe, we can have concurrent / parallel executions of the same code by 2 or 200000 invocations without adverse effects.

Applicability in meeting the challenges in scaling and performance

Today's applications need to fully utilize the underlying hardware. Moore's law doesn't holds when we consider raw cycle performance, but is still applicable if we consider the overall increase in processing capability through hyper threaded and multi core processor hardware. The information generation and its processing needs has increased too, leading to use of parallel computations. This is where the FP comes in handy. Its true that infrastructure can be abstracted from the application (as with cloud platforms) but if we do that ourselves, we can have greater flexibility and scalability for our solutions.
One feature of many 'newer' breed of FP languages like F# or scala that are hybrid in nature that I haven't discussed before is the ease of use of these languages is the ability to construct Domain Specific Languages; DSLs. From the business' perspectives, DSLs are fast becoming key to map the technology-business divide and keep application agility and re-usability simultaneously.

My exposure these days is with scala, which is a hybrid FP language that runs on JVM, and is interoperable with java. It is attracting the same kind of curiosity for java developers right now which ruby (largely because of Rails) did a few years back. However, this language aims at scalability, which is also its full name (scala stands for scalable architecture) within the JVM itself, making it an ideal candidate for solving middleware performance and scalability related issues. The fact that it replaced Rails in handling message processing in Twitter speaks for itself. I am also working on a research paper that explains the use of scala (and FP in general) to process massive amounts of data in distributed/cloud environment. Will post some more information here as it gets finalized.