Here are my comment on an often repeating topic of conversation about 6th semester projects in Masters Of Computer Applications course. A lot of people are considering programming for web based applications aka websites for their last semester projects in MCA. Since this involves a whole stack of technologies, one can observe that it in turn, helps a person or more importantly, his mentor to hide a poor design behind this complexity.
Coming back to the topic, a lot of my friends and people from various coaching centers have certainly got preconceived notions regarding different technologies to code with as their server side programming language. However, one can observe the faint rumblings of faint but still existent religious fervor of 'my language is better than yours'.
To start off with, students try and then ignore Java as an alternative because it is too complex (It is a big pain, but then has its benefits) and huge for their web applications. However, this fear can be allayed by the fact that one can choose from a wide variety of frameworks(MVC like Struts & SpringMVC or component based like Tapestry or JSF) and technologies (a plethora of middleware tech.) within the java ecosystem itself. PHP is touted as a compelling alternative, but for me, the scripting approach only works for demonstrative purposes. Anything larger either needs a huge patience, or frameworks, which are immature so far in this platform. I really am not impressed by frameworks like Zend here to name a few. Microsoft .NET deserves a special mention because it is already baked in (with IDE & app server ready, for instance) as well as has a meaningful architecture(through code behind). However the very strength of this framework becomes its nemesis as students are not encouraged to question/hack into the innards of any tool used. The upcoming framework like asp.net MVC too represents a copied and chaotic exercise to perform something that is coming too late.
My personal outlook about the whole situation is that no single technology rules the roost. If you ask me, it is java for building middleware (business intensive) & expandable (one where you can fit in a lot of components/features) websites. Php is really cute as you get the time to play around with css & javascript based functionality around your site (After all, the data is the king and your site functions as its glorified frontend). If I require a lot of versatility as well as ease of development, I'll go for .NET as it really hits that sweet spot when you are willing to work inside the confines of the framework.
I had this discussion with myself a few days back in a moment of clarity as I was contemplating mine and others' 6th semester project for my MCA course. I specifically ignored Rails (and frameworks based on it) as one generally threads a safe line in our course. Researching and hacking into a shiny new technology and writing a perfect but undocumented & incomplete project would only lead towards its cancellation.
BTW, I had time to write all this stuff on new year's eve as tomorrow is my 5th semester ERP examination. So good luck for me and a very happy new year 2011 AD for my readers!
Friday, December 31, 2010
Monday, December 6, 2010
The stuff behind the fluff – Is Green IT an exercise in vain?
Cloud computing is generally thought to be efficient, but this study changes it all. According to the recent UK study as published by DW World, cloud computing study reveals higher environmental impact than was previously thought. The quantity of the data centers as well as the consumers would collectively create this problem, which would be further intensified by the use of richer digital media. What this study surprised me was the analysis that it would be casual home users who would lead the excessive usage through rich media sharing/duplication.
In the past, Information Storage and Management used to focus solely on the enterprise specific needs. For handling the binary media format, technologies like CAS(Content Addressing Systems) are already in place. But according to this study, there would be a demand of 3200 Mega Bytes per person per day in just few years time. Given the pace of Internet proliferation and advancement, this is not a vague guess but a worrisome one.
The article pointed out the following courses of actions :
Creation of better/faster computers – This is really not possible as stated due to the lag between demands and Moore's Law
Green data centers – Again, here too the politics play an important role, BRIC nations in particular do have a cavalier attitude towards implementing clean sources.
What was missing here in my option was the point that the research was addressing the solution of the problem rather than its cause. So, instead of addressing the data storage issues, we need to rigorously safeguard against the redundancy in such data. Semantic web is one such approach. Other approaches such as tagging the digital media and having a single copy for identical media (different locations with same signature) could solve this problem. Also, instead of consolidization, the work on distributed computing should also be promoted.
The distributed computing can easily be done using map/reduce algorithms or through application development platforms like Apache Hadoop. The issue of data can be addressed using peer-to-peer data exchange as in bit-torrent. These wouldn't create problems themselves because of the ever decreasing costs of online bandwidth and its performance.
In the past, Information Storage and Management used to focus solely on the enterprise specific needs. For handling the binary media format, technologies like CAS(Content Addressing Systems) are already in place. But according to this study, there would be a demand of 3200 Mega Bytes per person per day in just few years time. Given the pace of Internet proliferation and advancement, this is not a vague guess but a worrisome one.
The article pointed out the following courses of actions :
Creation of better/faster computers – This is really not possible as stated due to the lag between demands and Moore's Law
Green data centers – Again, here too the politics play an important role, BRIC nations in particular do have a cavalier attitude towards implementing clean sources.
What was missing here in my option was the point that the research was addressing the solution of the problem rather than its cause. So, instead of addressing the data storage issues, we need to rigorously safeguard against the redundancy in such data. Semantic web is one such approach. Other approaches such as tagging the digital media and having a single copy for identical media (different locations with same signature) could solve this problem. Also, instead of consolidization, the work on distributed computing should also be promoted.
The distributed computing can easily be done using map/reduce algorithms or through application development platforms like Apache Hadoop. The issue of data can be addressed using peer-to-peer data exchange as in bit-torrent. These wouldn't create problems themselves because of the ever decreasing costs of online bandwidth and its performance.
Wednesday, November 24, 2010
Hurdles Towards Virtulization
Hurdles Towards Virtulization : What we face on its way to implementation
This post is dedicated to the current challenges that we face in the creation and use of virtualization. I am writing this even at a time when virtualization is something which is sorely missing in everyday computing and beyond for me. For instance, during the past few weeks, I had my both hard disks developing bad sectors and had a miserable experience upgrading Ubuntu on my friends' disk. We need a mechanism to seamlessly move the operating systems from one machine to another. For this, we need virtualization.
Given the current hype and hoopla over this term, many people believe that this is a recent phenomenon which is not the case. Virtualization has its roots at least 40 years before, with the concept of time sharing systems. Similarly, the IBM OS/360 also heralded this new era of 'virtual machines'(VM). In order to manage these Vms, we need a special program which is known as Hypervisor. Now the challenge with hypervisor is that it needs to provide a secure, equivalent, efficient and controlled resource environment for its guests(which are the virtual machines).
So what are the challenges ?
The first challenge that comes in my mind is the performance of the VM. If the guest is not getting enough processing, then there is no benefit out of such an arrangement. This can often happen when the guest OS performs a privileged operation such as a hardware interrupt or a system call resulting in acquisition of hardware locks. Now, as the guest has to run safely, this has to be carefully emulated by the hypervisor.
There is also a concept of time loss or clock skew. As there may be more than one OS per CPU running on a hypervisor at a given instance of time, the instructions by them would be run collectively. However, from the VM's perspective, there were only its instructions, so the instructions take longer time to execute (as other VM processes also hog the CPU cycles) than what the VM actually expects it to. Naturally, this time delay propagates to larger proportions leading to problems such as time-slicing of multiple processes at the guest OS/VM level.
The processing sharing problem also has a profound effect on the performance of the overall system. For instance, on a high priority guest, a low priority process would get precedence over a higher priority process on a low priority guest.This problem has not been solved to this date and only fringe works have been done in this regard. Perhaps this has still pertained due to the fact that a lot of OSs still are created with a single operator at a time without any restrictions.
The memory management as well as addressing are also some of the prevailing problems. Memory cache management is also a problem that rises exponentially as the number of intermediaries between the processes and hardware increase. This can be understood using a simple example. Let us consider that we have a machine which as LRU method of page catching and its guest has MRU method of caching. So as soon as the process becomes old in guest, it is discarded from the memory. However, on the host it is not deleted. So new we have wasteful data on the host memory. The other extreme of this situation is when the data is frequently required in this setup. Now, the host would start to thrash. We need to control this thrashing and ensure page faults remain constant and distributed over a period of time through 'handshaking' communication between the guest and hypervisor with regard to occurrence of faults.The final problem that plagues virtualization systems is the addressing of memory which is there due to different memory addressing schemes being followed in both the hypervisor and guest. The memory addresses are divided for processes into segments, pages and further into offsets. So every process has 2 addresses and 3 memory references, leading to extra overhead. This is avoided through a translation look-aside buffer (TLB), which generally stores the recently used segment and page addresses. But for virtualized environments, this isn't enough as in normal TLB translations, the virtual addresses are transformed into a real ones but due to 3 address levels (Actual hardware address, the 'emulated' hardware address for guest and the virtual address at the guest), there exists an overhead of extra translation. Thankfully, this is being addressed by hardware vendors through hardware support such as Intel's VT-i or AMD's NPT on one end and through utilisation of these enhancements by hypervisors such as virtualbox and vmware.
Hopefully in the near future, we can expect to see more prevalence of virtualized environments, not only in heavy duty systems, but on hardware ranging from the least expensive personal computing to mission-critical systems, given the increasing maturity of virtualization .
This post is dedicated to the current challenges that we face in the creation and use of virtualization. I am writing this even at a time when virtualization is something which is sorely missing in everyday computing and beyond for me. For instance, during the past few weeks, I had my both hard disks developing bad sectors and had a miserable experience upgrading Ubuntu on my friends' disk. We need a mechanism to seamlessly move the operating systems from one machine to another. For this, we need virtualization.
Given the current hype and hoopla over this term, many people believe that this is a recent phenomenon which is not the case. Virtualization has its roots at least 40 years before, with the concept of time sharing systems. Similarly, the IBM OS/360 also heralded this new era of 'virtual machines'(VM). In order to manage these Vms, we need a special program which is known as Hypervisor. Now the challenge with hypervisor is that it needs to provide a secure, equivalent, efficient and controlled resource environment for its guests(which are the virtual machines).
So what are the challenges ?
The first challenge that comes in my mind is the performance of the VM. If the guest is not getting enough processing, then there is no benefit out of such an arrangement. This can often happen when the guest OS performs a privileged operation such as a hardware interrupt or a system call resulting in acquisition of hardware locks. Now, as the guest has to run safely, this has to be carefully emulated by the hypervisor.
There is also a concept of time loss or clock skew. As there may be more than one OS per CPU running on a hypervisor at a given instance of time, the instructions by them would be run collectively. However, from the VM's perspective, there were only its instructions, so the instructions take longer time to execute (as other VM processes also hog the CPU cycles) than what the VM actually expects it to. Naturally, this time delay propagates to larger proportions leading to problems such as time-slicing of multiple processes at the guest OS/VM level.
The processing sharing problem also has a profound effect on the performance of the overall system. For instance, on a high priority guest, a low priority process would get precedence over a higher priority process on a low priority guest.This problem has not been solved to this date and only fringe works have been done in this regard. Perhaps this has still pertained due to the fact that a lot of OSs still are created with a single operator at a time without any restrictions.
The memory management as well as addressing are also some of the prevailing problems. Memory cache management is also a problem that rises exponentially as the number of intermediaries between the processes and hardware increase. This can be understood using a simple example. Let us consider that we have a machine which as LRU method of page catching and its guest has MRU method of caching. So as soon as the process becomes old in guest, it is discarded from the memory. However, on the host it is not deleted. So new we have wasteful data on the host memory. The other extreme of this situation is when the data is frequently required in this setup. Now, the host would start to thrash. We need to control this thrashing and ensure page faults remain constant and distributed over a period of time through 'handshaking' communication between the guest and hypervisor with regard to occurrence of faults.The final problem that plagues virtualization systems is the addressing of memory which is there due to different memory addressing schemes being followed in both the hypervisor and guest. The memory addresses are divided for processes into segments, pages and further into offsets. So every process has 2 addresses and 3 memory references, leading to extra overhead. This is avoided through a translation look-aside buffer (TLB), which generally stores the recently used segment and page addresses. But for virtualized environments, this isn't enough as in normal TLB translations, the virtual addresses are transformed into a real ones but due to 3 address levels (Actual hardware address, the 'emulated' hardware address for guest and the virtual address at the guest), there exists an overhead of extra translation. Thankfully, this is being addressed by hardware vendors through hardware support such as Intel's VT-i or AMD's NPT on one end and through utilisation of these enhancements by hypervisors such as virtualbox and vmware.
Hopefully in the near future, we can expect to see more prevalence of virtualized environments, not only in heavy duty systems, but on hardware ranging from the least expensive personal computing to mission-critical systems, given the increasing maturity of virtualization .
Thursday, October 28, 2010
Are we a dying breed ?
Is it really beneficial to build and maintain an online persona when it is fraught with problems all around the corner? This thought was echoed on a thread on javaranch.com, which happens to be a very popular social forum for java technology enthusiasts.
One of my acquaintance had also, quite recently expressed reservations about the same due to the lack of appreciation. Whether his comment is just his perception or is it the truth, remains to be seen. The concept of professional blogging too remains an elusive concept for me and as far as I have observed, this remains largely and exercise in isolation or through and 'tool'.
In the google trends on blogging, it appeared that the search volume index dropped to an all time low in the last quarter of 2010 ever since the usage curve became consistent since 2007. The reference of blogs in the news also became erratic and dropped during this period.
This leads to an interesting conclusion; is twitter really ushering in an era of microblogging since the people don't really want to go that extra mile for sharing their thoughts ? Applying social networking in the above comparison definitely causes no raised eyebrows as it is quite clear that social networking has taken the lead for the online citizens.
As the aftermath of this change, you can expect greater emphasis on the management of information over these platforms - both for commercial and research purposes as well as a general change in the dissemination of information. This is because instead if the traditional Author-reader/follower role, we're fundamentally shifting towards an peer to peer approach with spontaneous interactions. So instead of following your favorite blog and waiting for the posts to come in, real time and probably, geographically aware interactions rule the roost. Social networking today has spread much beyond socializing with friends, family and that 'hot chick' profile(pun intended) to an information sharing portal. It is customizable as a mini internet with all the users being able to be the generators of the content.
So if you are a techie or business savvy, you can utilize this change for your benefit. What do you feel about this ? are there any new developments in this arena that you are aware of ? please do comment about it.
One of my acquaintance had also, quite recently expressed reservations about the same due to the lack of appreciation. Whether his comment is just his perception or is it the truth, remains to be seen. The concept of professional blogging too remains an elusive concept for me and as far as I have observed, this remains largely and exercise in isolation or through and 'tool'.
In the google trends on blogging, it appeared that the search volume index dropped to an all time low in the last quarter of 2010 ever since the usage curve became consistent since 2007. The reference of blogs in the news also became erratic and dropped during this period.
This leads to an interesting conclusion; is twitter really ushering in an era of microblogging since the people don't really want to go that extra mile for sharing their thoughts ? Applying social networking in the above comparison definitely causes no raised eyebrows as it is quite clear that social networking has taken the lead for the online citizens.
As the aftermath of this change, you can expect greater emphasis on the management of information over these platforms - both for commercial and research purposes as well as a general change in the dissemination of information. This is because instead if the traditional Author-reader/follower role, we're fundamentally shifting towards an peer to peer approach with spontaneous interactions. So instead of following your favorite blog and waiting for the posts to come in, real time and probably, geographically aware interactions rule the roost. Social networking today has spread much beyond socializing with friends, family and that 'hot chick' profile(pun intended) to an information sharing portal. It is customizable as a mini internet with all the users being able to be the generators of the content.
So if you are a techie or business savvy, you can utilize this change for your benefit. What do you feel about this ? are there any new developments in this arena that you are aware of ? please do comment about it.
Tuesday, October 12, 2010
From theory to practice- Transitioning isn't as easy as it seems
I've been involved recently with the task of creating different projects. In order to do this, I followed the simplest and most important rule in the book – just put the most dedicated individuals on your team. However, with the induction of each separate member in the team, the allocation of responsibilities became more complex and the communication too started becoming a thorn in the flesh for the project vision that I had in mind. However, through agile approaches, currently I am able to lead and integrate my team effectively.
I didn't had enough time to bask in the glory of my newly-founded team equilibrium when my junior students asked me to mentor for their project too (we're having a common project by IBM, namely TGMC 2010). Okay, I said, and went into their class for a briefing. Upon entering the class, I was surrounded by a group of around 70 students, all willing to enter (and most of them already have) in a contest requiring people to submit Ajax driven websites that leverage SOA as their middleware, but very few having experience in even basic html . To get the people up and running, I created a blog wrote a basic how-to (http://bbdnitm-tgmc.blogspot.com/2010/10/getting-up-to-speed-towards-building.html).However, after 4 days, I am yet to get a comment from these people and have resigned to the fate that they need to work really hard if they ever want to see anything at the end of the tunnel. A few of them might deliver a workable solution after 6 months, but for the rest of them, the pain would be inevitable. From my side, I am willing to mentor any person who is a quick learner of jsp and middleware stuff and although I don't know much about the IBM's product usage, I am willing to learn and help others ease into deployment into IBM specific technologies like DB2 and Websphere/RAD. This incident has led me to think about project management in a new light, and now what I am feeling personally is that it is as important to have an able team as it is to find a mentor. Apart from this, it is the approach that matters for an individual, rather than his/her skills.
I didn't had enough time to bask in the glory of my newly-founded team equilibrium when my junior students asked me to mentor for their project too (we're having a common project by IBM, namely TGMC 2010). Okay, I said, and went into their class for a briefing. Upon entering the class, I was surrounded by a group of around 70 students, all willing to enter (and most of them already have) in a contest requiring people to submit Ajax driven websites that leverage SOA as their middleware, but very few having experience in even basic html . To get the people up and running, I created a blog wrote a basic how-to (http://bbdnitm-tgmc.blogspot.com/2010/10/getting-up-to-speed-towards-building.html).However, after 4 days, I am yet to get a comment from these people and have resigned to the fate that they need to work really hard if they ever want to see anything at the end of the tunnel. A few of them might deliver a workable solution after 6 months, but for the rest of them, the pain would be inevitable. From my side, I am willing to mentor any person who is a quick learner of jsp and middleware stuff and although I don't know much about the IBM's product usage, I am willing to learn and help others ease into deployment into IBM specific technologies like DB2 and Websphere/RAD. This incident has led me to think about project management in a new light, and now what I am feeling personally is that it is as important to have an able team as it is to find a mentor. Apart from this, it is the approach that matters for an individual, rather than his/her skills.
Subscribe to:
Posts (Atom)