On why we fail at DevOps

Seeing that devops practice does not really seems to be taking off in my present workplace, I decided to decode the underpinnings of the concept and try to interpret what is perhaps not going in the desired direction and more importantly, what is required to be applied which will enable this transformation.
In different texts, I came across a common theme that covered DevOps in detail and each pointed out to The 3 principals underpinning the devops practice [http://itrevolution.com/the-three-ways-principles-underpinning-devops/]

Arguably, DevOps is about making that process of build/release transparent and automated to make this process smoother, but just like agile, it is more than just tools (ant/maven/, jenkins, sonar, docker,etc.) but is more of the discipline that needs to be imbibed. Few years back, most of the server side application developers used to treat UI like anathema but due to rise of Javascript as a first class language, one can not longer resist being a full stack developer. In somewhat a similar discipline, there is a need for developers to treat build configuration (gruntfile, dockerfile, etc) as a vital portion of codebase and likewise for system administrators, who also require repository access to check-in their changes.

In monolithic softwares like SAP Hybris, directly building docker images are a challenge at the moment as only the datahub, essentially a tomcat web application can easily be dockerized. But it remains to be seen whether we will have more microservice/YAAS based approach in the post 6.3 releases of Hybris.

my two cents

JRebel alternatives for SAP Hybris

Developers hoping to accelerate their Hybris debugging time while skipping the time consuming steps of ant clean all - server start and then debugging can do so with DCEVM and change code on the fly basis and view it on the browser without having to restart hybris.

Advantages

• Reduced time to carry out trivial changes.
• Faster development cycle.
• Flexibility in trying out features.
• Increased server uptime.

Step-by-step guide

Login into help.hybris.com and view the following page for this:
https://help.hybris.com/6.2.0/hcd/0b40907d5db14955bf4074b4b25e6998.html

Although the steps are mentioned in the hybris help, this post is done to incorporate other changes that we faced here in Eperium.

1. Download the light installer from https://github.com/dcevm/dcevm/releases
2. Open cmd from start menu and right-click 'Run as Administrator' (as there are no sudo user console otherwise in Windows)
3. cd to location where DCEVM jar is present and run java -jar DCEVM-light***-installer.jar 
4. Run the install DCEVM as altjvm for your JDK
5. If there is an permission error, you need to check if jar was run with admin privledges (see point #2 above)
6. If there is an error 'could not get dceversion of ..' , then download a different version (try older and stable versions from step #1 above)
7. Add the following to local.properties

tomcat.debugjavaoptions=-XXaltjvm=dcevm -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=n

Do an ant to update configurations in the embedded tomcat server (used by Hybris) and run Hybris server in debug mode. Now connect Eclipse with it as remote process.

Migrating Hybris with Ease

Running Hybris On Oracle - and deriving advantages of its data backup/recovery mechanism.

During a common course of Hybris implementation, at times changes made (intentionally or otherwise) in the schema/data can adversely affect the development environment and make it unstable. In some cases, initialization of system is the only resort which means losing all the manually created configurations that are made in the system and re running the impex scripts (which are not hooked up for their automatic execution). However, this is time consuming as well as requires manual intervention if the impex scripts and external settings are not properly set. Similar in design of rails active record where the database state is kept in form of migrations, the data structure is kept under impex and items configuration and updated as necessary.

To alleviate this problem in our latest project, we are using the same oracle database that is present on the staging server and just after re-initialization, resetting the database back to its saved state.
Periodically, the database team resets the master db backup to keep up with the occurring schema changes (which ensures that the additional work that a developer has to perform after 'initializing' the system is kept to minimum). So instead of running all the impex and doing synchronization, only a typical system update suffices.

Here's a short description of how the database is reset from its backed up state.


1. Disconnect from all running applications/programs that oracle user is logged into.

2. Log on to SQLPLUS as admin user: sqlplus / as sysdba

3. Drop and Create the user via the sql file with the following text (the path where the dbf files are present inside the oradata folder).

CREATE TABLESPACE YHTSPACE DATAFILE 'G:\app\oracle\oradata\HYBRIS\YHSPACE.DBF' SIZE 20M AUTOEXTEND ON LOGGING;
CREATE TEMPORARY TABLESPACE YHTEMP TEMPFILE 'G:\app\oracle\oradata\HYBRIS\YHTEMP01.DBF' SIZE 5M AUTOEXTEND ON;
define us=aphy
define pw=aphy
define ts=YHTSPACE
define tts=YHTEMP
DROP USER &us cascade;
CREATE USER &us IDENTIFIED BY &pw DEFAULT TABLESPACE &ts TEMPORARY TABLESPACE &tts PROFILE DEFAULT ACCOUNT UNLOCK;
GRANT "CONNECT","RESOURCE","CTXAPP" TO &us;
GRANT UNLIMITED TABLESACE TO &us;
ALTER USER &us DEFAULT ROLE ALL;
GRANT CREATE CLUSTER TO &us;
GRANT CREATE DATABASE LINK TO &us;
GRANT CREATE SESSION TO &us;
GRANT create ANY INDEX TO &us;
GRANT CREATE SEQUENCE TO &us;
GRANT CREATE SYNONYM TO &us;
GRANT CREATE TABLE TO &us;
GRANT CREATE VIEW TO &us;P
GRANT CREATE PROCEDURE TO &us;
GRANT CREATE TRIGGER TO &us;
GRANT CREATE TYPE TO &us;
GRANT CREATE SNAPSHOT TO &us;
GRANT EXECUTE ON CTX_DDL TO &us;
GRANT ANALYZE ANY TO &us;
grant IMP_FULL_DATABASE to &us;

4. Run the import command:
imp aphy/aphy FROMUSER=APQA TOUSER=aphy file=G:\app\oracle\oradata\HQD_22072016.dmp


While this method of managing database is time saving, it inherits the problem found in most of the tailor made approaches which involve manual intervention. Whether this is a worthwhile approach or a poor one is totally up to you and can be used in a manner depending upon your needs.

Marching towards Microservices

To say that microservices have arrived would be the understatement of the year 2015 when most of the enterprise application creators seem to have realized the importance of keeping individual parts of application separately.
There is nothing new when it comes to microservices and there are tons of literature - ranging it from precise definitions stated by Martin Fowler, to the loosely coupled architectural term.

It is also easy to confuse microservice with SOA as the design seems similar; however, SOA is an integration of different applications, but microservices remain independent to each other - you can simply plug in and out different services while using them across different applications.

The  rise of javascript based frontend for web applications and in general, polyglot programming of applications has further argued the premise of having standalone pieces of applications that can

A) Scale very well

The individual service can scale up easily - as its underlying micro-architecture contains the least surprises and given today's virtualized/cloud environments, such services can be increased in volume as well as tested and rectified if something goes wrong.

B) Standalone

The services are standalone in the sense that each application that uses a microservices can use it with a specific set of data and as applications interact over http, there are no hassles of contract based services over SOAP/REST based services, which were based on service contract and were the norm earlier. So, we do not need an elaborate contract or some specific tool ; just a miniature web application framework would do.


An example of microservices making waves is into the SAP Hybris ecosystem via the YAAS that serves both as a place to host as well as market individual microservices for different e-Commerce implementations. Hosting is important because of the sensitive nature of data of different clients (which can be stored as different tenants) and also to remove the complexity of managing the server load when there are a lot more things to be completed.

Creating standalone applications and utilizing their mashup is no longer a field dominated by selected few, but are quickly becoming the norm in the coming  time and is poised to become the disruptive force of the decade.

True Omni-Commerce with IoT

The changing face of e-commerce means that successfully buying a product is more than just a few clicks away. Today, with the widespread influx of communicative devices and advertisements, the customer is constantly bombarded with advertisements of interest right from the morning tea to the evening commute, everywhere advertisements try to sell different products. However, this might only affect the user decision, which might be in the favor of competitors of the advertiser. So the context of advertisement with respect to the end customer needs to be revised and the user needs to be engaged -  buy the product rather than just influence decision making. SAP Hybris is currently aiming towards creating the online storefront of the future and is tinkering with some of the things that I am discussing- but a lot needs to be done before things are

Internet Of Things

As defined in Wikipedia, "The Internet of Things (IoT) is the network of physical objects or "things" embedded with electronics, software, sensors, and connectivity to enable objects to collect and exchange data."
Smart devices/sensors are already increasing exponentially and are poised to make the next big change in everyday life since the rise of internet and by 2020, only the B2B based such devices would be around 5.4 Billion.

Applied IoT

We've all heard about Amazon drones but the ideas do not stop there; by making a user context aware, the e-commerce can jump out of browser to mobile notifications through NFC enabled portals in physical stores. On the other hand, the user based sensors could be used for kiosks and physical display stores, making them intelligent. Another scenario could be the use of RIFD enabled tags on consumables, which can enable gamification based points for repeating customers, leading to repeated orders. These ideas are dime a dozen, but it remains to be seen which technology and business goes all out to adapt them.

True Omni Commerce

Although this sounds good in theory, in real world things like scalability and security take precedence and instead of just improving the context, there is a need to provide actual sales through these additional channels, or in other words, creating a mashup involving existing channels to generate even more avenues. I am currently working as a Hybris developer and am hoping to see some positive influences due to rise of such a technology, which is currently in the playground stage.

My tryst with Hybris... continued

Rather than making a new post on my thoughts about development over hybris platform, I felt prudent to continue my older post about SAP Hybris and bring out some of the challenges felt and my take on this e-commerce platform.
Currently, I am still a newbie developer working on various features and parts of the platform.
My team is also in the young hybris phase as explained in detailed manner here ( http://time2know.net/young-hybris-partner ) and the knowledge sharing is limited and as frustrating as it seems, the development work is generally slow due to long list of assumptions that hybris platform assumes.

One example is in implementation of AdminCockpit - where all the fields in various editors are laid out in various xml files:

< editor >
    < group qualifier="General" visible="true" initially-opened="true" >
        < label >General< / label >
        < property qualifier="B2BRegistration.title" / >
        < property qualifier="B2BRegistration.name" / >

These files are editable, but changing them yielded nothing - and no suitable help was found in hybris wiki or experts forum. Next , I tried doing ant all and searching for generated files but found nothing of importance. Finally, I tried doing an initialization of cockpit and admin cockpit which made the changes appear - the amount of efforts that I invested were shockingly huge as compared to the actual work needed.
If some hybris expert/creator reads this, I hope my plead for greater technical and functional documentation reaches - debugging without a knowledge of how the APIs function is like learning to swim inside a boat.
Development and trying to appear for hybris certification remain my top priority missions for the coming month and I hope to be a hybris certified developer after then and work on this platform for the near future.

Using Hybris for e-commerce

Recently, I had a chance to work on Hybris, which is a private company (now acquired by SAP) offering an application framework with the same name, which is a customizable platform for handling e-commerce B2B and B2C needs. While I was initially skeptical of the framework in the sense that it is not apparently clear how to create an online store like Magento or Spree, but is much more than a CMS that creates a shopping front-end for the user.
Technically, hybris is a framework built on top of spring that runs on a customized tomcat or SpringSource DM Server and uses maven for build automation. While this architecture is carefully thought out, the problem lies with its openness- the community is quite limited and as learners, we are restricted to the hybris wiki and forums. Given the evolving nature of the application (different versions come frequent and fast), opening it up would make a lot of sense. As SAP recently acquired Hybris, it may provide integration with its tools and databases in future as well as provide forums and community support like other mainstream software.

Coming back to uniqueness in this framework, here are a few notable observations:
Open-Close model: While the software is not open source, the way of creating application on top of framework comes with least surprises and there is a lot of flexibility.
User oriented: Cockpits are specialized interfaces that power users/admins of the software can use to quickly access the information present in their application.
Thought out: Like a matured product, entire gamut of e-commerce application is present and one can not only provide the http based web solution but also plugin with an existing application.
Scalable: As jvm based approach is followed, it is quite scalable - though it requires an upfront resource allocation during development, it sure pays back to ensure the scalability of the website.
Performance: The customization comes at a cost of computing resources, as well as slow/complex process workflows. The overall performance of the application if compared against other ecommerce solutions is worse.
XML based: The configuration is mainly in xml and parts like the UI is built on custom framework, which is pretty restrictive. While these might not be the issues, but they stuck out like a sore thumb to me, which brings us to another conclusion discussed in the next point.
Old standard Architecture: Application development seems archaic on this framework as the choice of the technologies used as well as provided seems to be a decade old - and regardless of the things mentioned on the website, integration with popular technologies is quite hard.

As I continue my exploration into this framework while working on real world projects, the things gained so far would help me to compare this against both open-source and corporate alternatives and enable me to contrast the differences more accurately against each of them.