Repairing your problematic display drivers

I am sharing a recent experience with my laptop. It is a MSI CR-500 that houses an Nvidia GeForce 8200m G graphics card. In previous Ubuntu(11.04 and earlier) versions, the stock graphics driver resulted in suboptimal displays and compiz and other desktop polishing tools were not able to be installed.

Out of habit, I tried to install a custom Nvidia binary driver into my newly installed Ubuntu 12.04 last week but the graphics driver failed to install correctly. The newer linux kernal did not allow my exsiting drivers to install so had to download a newly released driver. Undaunted, I tried installing a new driver, but it failed and my graphics display manager went kaput.

To get the things back, I simply reinstalled the Ubuntu and got my settings back. Here's what I did:

Booted off 12.04 livecd/pen drive

• In install option window, went to manual drive partitioning option
• Selected the partition containing the 12.04 and set its mount point to /
• Did not select the format disk option – to retain my earlier settings and installations.

It gave me a warning specifying deletion of files.

The OS got restored in 20 minutes and I was good to go!

NOTE : Nothing much got deleted as I was having a fairly new OS with not much loaded components. Surprisingly, the java7 unzipped at /usr/local/lib was untouched, but the scala installed at /home/sumit was removed.

Practical Malware Analysis: Book review

Practical Malware Analysis: Book review

This is my review of the book, practical malware analysis by Michael Sikorski and Andrew Honig done  under the Oreilly Blogger Review Program.

This book teaches you the techniques and strategies followed by professionals to analyze and identify malware. As windows continue to be the most used OS in the world, it is not surprising that malware ranging from annoying worms to cyber weapons like stuxnet continue to spread using different means over the windows operating system.


Being a security book,  I was looking forward to a lot of exercises and security tools that would assist me in finding details about the malware that I might require. The book does the necessary job but often strays off its topic as it delves into the basics for what is more than sufficient, creating discontinuity in reading the text.

Tools such as OllyDbg, IDA pro, Win Dbg,etc are given in sufficient detail and various chapters are dedicated to their various uses in analysis and reverse engineering, which will be beneficial to an security professional. From a casual user point of view, the expansive details might be more of theoretical annoyance and the book is at places too advanced into the details.

On the nicer features in this book, there is a keen focus on practical implementation of the things taught at the end of each chapter in form of a set of labs that the end user is expected to complete. For me, this worked very well as I was able to skim across various chapters and perform lab routines to reinforce my understandings.

One of the caveats of having an extended introduction of various terminologies is that they seem stretched a bit too long. The book deals almost exclusively with the windows OS, so the name of the book should've been Practical Windows Malware Analysis which would aptly reflect the target environment of the book. As a user, it was a rewarding experience in reading from the book if the order of chapters were followed and the lab exercises done.

Python coding conventions

It is often a confusion in my team regarding what practice to follow while coding in python.
I have research it (mainly from PEP-08) and am sharing my findings.
Code Style
==========
Idioms
::::::
Zen of Python (PEP-20)
----------------------
Type import this on python shell to see the zen of python.

The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!


PEP-8
=====
Naming Conventions
------------------
As the naming conventions of python are otherwise loosely defined, some rules were created to be followed by all standardized modules.
In this, the python community is undecided. Generally, the java convention is followed, but with an underscore instead of capital letter between letters in a word, but capital case as in C# is also observed.

Underscore usage:
-----------------
This is not a syntactical requirement, but a convention.
 _single_leading_underscore
 weak "internal use" indicator. Eg; from M import * does not import objects in M which start with an _
 single_trailing_underscore_
 used by convention to avoid conflicts with Python keyword. Eg; class_ m


Abbreviations should be capitalized like URLBroadcastListener



Code lay-out
------------
Use of 4 spaces per indentation level, avoid tabs and do not mix tabs with spaces.
Do not exceed 80 character lines.
Avoid compound statements (single line having more than one line of code separated by a semicolon ; )
Blank lines can be used to perform the following:
To separate logical spaces(only if necessary) within the methods.
To separate different methods within a class.
Class definitions and functions may have 2 blank lines in between.

Imports
-------
Each should be on separate lines, but could be supplied on the same line if presnet in same namespace.
Order of imports:
----------------
  Standard imports
  Related third party imports
  Local application/library imports
Use absolute imports from within the namespace instead of relative values.
Specify clearly, if using a module containing a class as: from namespace.classname import classname to reduce ambiguity

WhiteSpaces
-----------
Whitespace formatting is best left to our own descretion, and should make the code readable.
Reiterating this often forgotten best practice- Never mix tabs with spaces.
Avoid WhiteSpaces at:
Just after start of parenthesis or a method call. eg: def meth ( arg): #Both the spaces around (
Just before a comma, semicolon or a colon. eg: def meth(arg , arg2) :
Just before indexes. eg: collect ['paper']
More than a single whitespace around operators. eg: pi  =  3.14159
No whitespaces should be used around keyword arguments or in assigning default parameter values
Compound statements on the same line (separated with a ; or : ) are discouraged as they affect readability.


Comments & Documentation strings
--------------------------------
Here too, common programming rules apply: Comments should only be used if code is not easily decipherable; and should be updated with code changes.
Comments should be complete line in itself with longer ones finishing with a period (.)
Block comments start before any block of code with a single # and a space.
Inline comments should be rarely used, if that particular line performs something critical for the developer's attention.
Documentation strings (aka documentation comments in some languages) should be written for all public classes, methods, functions and modules.
These should preferably be in multiple line.

PEP-257
=======
Docstring Conventions
---------------------
These conventions are optional, as they pertain to the documentation portion of python. Still, these can be quite handy if you would be releasing the api documentation of your application later as api generators like Docutils require that you conform to a specified convention.
Docstring is a string literal occuring at the first statement of a method or class.
It is created using three " literals and can be single or multi line.

It is worth noting that the zen of python must take precedence over yours or mine preferences. This is necessary to allow programmers from diverse backgrounds to communicate effectively in code (ours was a mix of java, C# and visual basic programmers and we were using a wrapper of a C++ library, leading to major headaches and refactoring towards code maintenance).

"Programming Entity Framework DbContext" by Julia Lerman & Rowan Miller; O'Reilly Media

This is my review of the book, programming entity framework DbContext by Julia Lerman & Rowan Miller, done under the Oreilly Blogger Review Program.

Targeted at developers familiar with, and using entity framework in their applications, especially at places where object context interacts with the core of the framework, this book offers developers all about DbContext API, with the how-tos for dealing with creating customized queries to the database when the built-in mechanism of the framework does not suffice. Julia Lerman is the leading authority on learning Entity Framwork and Rown Miller is the project manager in Microsoft's team of ADO.NET Entity Framework.



The point of having this book is simple, after all that ease of creation of database entity models (and their corresponding configuration files) , one needs to interact them with application code where middle tier code performs interactions with the entities.

The book jumps into the differences between DbContext and ObjectContext at the start and explains the download instructions as well as code examples explaining integration with outside code as well as edmx model designer. It then explains the query part  and demonstrates LINQ queries to interact with the entities using DbSet in an eloquent manner while explaining entitiy retrival in detail (something that is rare to find by in a .NET text). I felt at ease with the details and quality of the coverage of the DbContext API and it almost felt like an in depth coverage into an established ORM tool like Hibernate or JPA while dealing with entities and relationship sets.It then proceeds to explain problems and solutions regarding tracking change of entities without the availability of a database, as the disconnected context state management is a feature which is hard to figure by itself in n-tiered applications.

The second half of the book deals with the newly released contents in the entity framework. It covers change tracker in detail and provides ample code along with the details of the new api. The validation api is also new in the framework and is introduced comprehensively, and some of the examples that I attempted worked correctly. The customization of validations is also covered in detail which is very handy to use in practical applications. The chapter on advanced features is mostly about unit testing with dbcontext, and should be named so instead of its generic name.
The book finishes off with the insight about the upcoming entity framework 5, but lacks any code example as the beta for the same is not out.

Overall, the book follows a style quite similar to a cookbook, but packs in with some well guided theory as well. The lack of appendix is missing, which could be there instead of the last chapter but the rest of the book follows the contents around the topic and is indeed a pleasurable learning experience. For developers looking forward to another .NET book with extensive visuals and wizards, this would come as a disappointment but for those looking to solve their middle tier infrastructure plumbing with proper code, this is the solid reference to  be kept near while developing.
You can purchase this book from here.

Configuration Best practices in Python

Based on my research over different ways of saving the configuration data of an application,
I have come across some interesting ways of saving the configuration.

ConfigParser (known as configparser in python 3 )
The ConfigParser class implements a basic configuration file parser language which provides a structure similar to .ini files in windows.

Suppose, I am having the following configuration file:
[Section]
java_home: /usr/java/jdk1.7.0
home = /home/sumit
something: this is a multiple line statement
    indented on each new line.

To read this, I can have the following :

import ConfigParser, os
# create a basic configuration parser
config = ConfigParser.ConfigParser()
#Use it to open our config file
config.readfp(open('config'))
# Read under the section, section for the first(0th) value having the key as home
content =  config.get('Section', 'home', 0)
print "received: "+str(content)
content =  config.get('Section', 'java_home', 0)
print "received: "+str(content)
content =  config.get('Section', 'something', 0)
print "received: "+str(content)


Writing into a config parser is equally easy, we simply insert the desired values into the config

import ConfigParser, os
conf = ConfigParser.RawConfigParser()
conf.add_section('my section')
conf.set('my section',  'name', 'Ganesh')
conf.set('my section',  'bool', 'true')
conf.set('my section',  'percentage', '65.34%')

#saving our changes into a configuration file, finally
with open('output.cfg',  'wb') as configuration_file:
    conf.write(configuration_file)

There are still other libraries that provide more functionality for the saving of such configuration data, such as http://cfgparse.sourceforge.net , or some of the ones are build on top of ConfigParser http://www.cs.wisc.edu/~param/software/cfgparse . Numerous other libraries also exist for different/customized solutions for the same issue.



Binary parsing of data
This is the serialization of the data (or in simple words, the flattening of different forms of data in binary format)
We can use pickle, or its C-based implementation, cpickle in order to save the data faster.
this is helpful if we are trying to save or load a large amount of data, or this process has to go on repeatedly in a short amount of time.

We bind the data through the following routine:
import pickle

data  = 'this is some form of data to be persisted'
list = [1,2,3,4,5]

opfile = open('data.dbi',  'wb')

# pickle the textual data using the protocol 0
pickle.dump(data, opfile, 0)

# pickle the list data using the highest possible protocol
pickle.dump(list, opfile, -1)

opfile.close()

This creates a binary file, data.dbi which has the binary data.
To recover the data back again from this file, we reverse this process and use pickle again to reclaim the original data.

import pickle

#unpickles the data back from the serialized file

pickle_file = open('data.dbi', 'rb')

data1 = pickle.load(pickle_file)

print(str(data1))

data2 = pickle.load(pickle_file)

print(str(data2))

pickle_file.close()

Thus, we have different ways of performing the same data, while following the best practices to cater to the needs of viewing the configuration by the end user or to ensure efficiency in processing of the data.