Archive for the ‘Ani's Blog’ Category

Continuous Integration and Performance Management

Wednesday, March 18th, 2009

This is my first post in many ways;
1. It is certainly the first one of the year 2009 (shame!). By the way “Happy New Year”!
2. I don’t remember when was my last blog post (shame again!!).

I have got a perfectly valid set of arguments to justify “me not blogging”, but I would not want to bore you lot with that ;) Instead, let’s talk about something interesting, like “Continuous Integration”! I am sure you all do it and so do I. We @Cake use Atlassian Bamboo as our CI tool, backed up by Ant + Ivy or Maven2 build infrastructure using JUnit. On top of that we have an excellent process of ‘Code Review’, which ensures that we deliver software of a very high standard. In other words, we are perfectly happy with our unit tests running overnight, showing a ‘green bar’ and developers addressing their code reviews first thing in the morning. I just love the ‘green bar’ though, but that’s besides the point. Now the question is “Is that enough of Continuous Integration?” or “Does that describe a fairly complete CI process?” I would have said “Yeah, thats totally enough”, till 28th January and I am sure many of you would still say that! But, things did change a bit for me since 28th January 2009, as I attended a webcast from DeCare Systems, titled ‚”Introducing CPM Toolkit ‚ Bringing Continuous Performance Management to JProbe”.

I have to agree, that I was a bit skeptical about the webcast. I was thinking in true Dilbert style, that “I am an engineer, I can’t listen to an hour of marketing talk”! But, boy oh boy, I was wrong! Not only did I like the webcast, I have also been taken by the “Continuous Performance Management” idea (honestly, it’s new to me). Jason Berry, the lead architect of CPM Toolkit did a wonderful demo. The tool is basically a combination of Continuous Integration and Performance Management. Which brings Continuous Performance Management Toolkit, a.k.a CPM Toolkit into play.

Here you can find a brief introduction of CPM Toolkit. Among others, the two points that got me hooked into it are, “Discovering performance problems when they are cheapest to solve” and “More time resolving and less time searching for the performance problem”. The CPM Toolkit allows you to pin point the performance problem at a unit test level, so you have a lot less code to find the problem in. This gives you the opportunity to quickly fix the problem and verify that your solution is not causing performance problems for any other parts of your system. I am pretty impressed with the GUI work the guys at DeCare has done. The graphical representation is simply impressive. To get a better idea of how CPM Toolkit works, I suggest reading Jason’s blog about CPM in practice.

One point I must mention here that I am still not 100% convinced that the use of yet another tool in the CI process will actually cut down development cost. Mainly because, I still need to see a real enough example of CPM Toolkit at play. So, I have decided that I am going to try out CPM Toolkit on my next project to get a better feel of it. I will definitely blog about the experience. In the mean time I would advice you guys to get a Free trial evaluation and give it a go!

I almost forgot that I still don’t have an answer to my initial question of “A perfect Continuous Integration process”. I guess, it’s very difficult to give a general answer which will fit all organizations. It’s more to do with the people involved in the process and the tools that are used. There will always be the next new cool tool, but it needs to be judged whether that tool fits into the process or not. I think it is fair enough to say that, introducing CPM ToolKit to your existing CI process can only make it better!

PostgreSQL database backup and restore

Sunday, February 24th, 2008

In the last couple of weeks I have been using PostgreSQL quite a lot. I have used PostgreSQL before but this time it was different as the development as well as the deployment environment were Linux based, as a result I was denied of the use of pgAdminIII, the GUI based administration tool for PostgreSQL database. My only option was to use the command prompt.

The need of the hour was to backup the live database and restore it on the test environment, which incidentally was also a Linux box. With pgAdminIII, it would have been really simple, take a backup of the database in a binary format and restore the test database from the binary file. So, off I went, digging into the PostgreSQL documentation looking for the commands which will do the same for me and wolla, I found pg_dump and pg_restore.

pg_dump is the utility for backing up a PostgreSQL database. Following is the command with it’s options that will create a binary backup of an entire database;
pg_dump -i -h host-name -p port-number -U username -F c -b -v -f "backup-filename.backup" dbname

pg_restore is the utility for restoring a PostgreSQL database from an archive created by pg_dump in one of the non-plain-text formats. Following is the command with it’s options that will restore the database from the binary backup file;
pg_restore -i -h host-name -p port-number -U username -d dbname -a -v "backup-filename.backup"
I noticed an interesting point while using pg_restore. pg_restore doesn’t work properly if there are referential integrity among the tables on the target database as tables are restored sequentially. So, the best restoring approach is to first create the tables in the database without enforcing their referential integrity, then run pg_restore to perform the magic. Once restoring is complete you can easily run another script to enforce all the referential integrities at one go.

On a completely different note, it is always considered as a best practice to test your application with a real data-set. This will give you the opportunity to write off those final few bugs which are only exposed when the application is used with a real set of data. What else could be better than the replica of the live database! More over, now you can backup and restore entire PostgreSQL database, even without the help of pgAdminIII.