Archive for March, 2009

Continuous Integration and Performance Management

Wednesday, March 18th, 2009

This is my first post in many ways;
1. It is certainly the first one of the year 2009 (shame!). By the way “Happy New Year”!
2. I don’t remember when was my last blog post (shame again!!).

I have got a perfectly valid set of arguments to justify “me not blogging”, but I would not want to bore you lot with that ;) Instead, let’s talk about something interesting, like “Continuous Integration”! I am sure you all do it and so do I. We @Cake use Atlassian Bamboo as our CI tool, backed up by Ant + Ivy or Maven2 build infrastructure using JUnit. On top of that we have an excellent process of ‘Code Review’, which ensures that we deliver software of a very high standard. In other words, we are perfectly happy with our unit tests running overnight, showing a ‘green bar’ and developers addressing their code reviews first thing in the morning. I just love the ‘green bar’ though, but that’s besides the point. Now the question is “Is that enough of Continuous Integration?” or “Does that describe a fairly complete CI process?” I would have said “Yeah, thats totally enough”, till 28th January and I am sure many of you would still say that! But, things did change a bit for me since 28th January 2009, as I attended a webcast from DeCare Systems, titled ‚”Introducing CPM Toolkit ‚ Bringing Continuous Performance Management to JProbe”.

I have to agree, that I was a bit skeptical about the webcast. I was thinking in true Dilbert style, that “I am an engineer, I can’t listen to an hour of marketing talk”! But, boy oh boy, I was wrong! Not only did I like the webcast, I have also been taken by the “Continuous Performance Management” idea (honestly, it’s new to me). Jason Berry, the lead architect of CPM Toolkit did a wonderful demo. The tool is basically a combination of Continuous Integration and Performance Management. Which brings Continuous Performance Management Toolkit, a.k.a CPM Toolkit into play.

Here you can find a brief introduction of CPM Toolkit. Among others, the two points that got me hooked into it are, “Discovering performance problems when they are cheapest to solve” and “More time resolving and less time searching for the performance problem”. The CPM Toolkit allows you to pin point the performance problem at a unit test level, so you have a lot less code to find the problem in. This gives you the opportunity to quickly fix the problem and verify that your solution is not causing performance problems for any other parts of your system. I am pretty impressed with the GUI work the guys at DeCare has done. The graphical representation is simply impressive. To get a better idea of how CPM Toolkit works, I suggest reading Jason’s blog about CPM in practice.

One point I must mention here that I am still not 100% convinced that the use of yet another tool in the CI process will actually cut down development cost. Mainly because, I still need to see a real enough example of CPM Toolkit at play. So, I have decided that I am going to try out CPM Toolkit on my next project to get a better feel of it. I will definitely blog about the experience. In the mean time I would advice you guys to get a Free trial evaluation and give it a go!

I almost forgot that I still don’t have an answer to my initial question of “A perfect Continuous Integration process”. I guess, it’s very difficult to give a general answer which will fit all organizations. It’s more to do with the people involved in the process and the tools that are used. There will always be the next new cool tool, but it needs to be judged whether that tool fits into the process or not. I think it is fair enough to say that, introducing CPM ToolKit to your existing CI process can only make it better!

‘Oracle Database Security and Compliance’ Conference

Tuesday, March 3rd, 2009

I have attended Oracle one-day conference titled ‘Database Security and Compliance’ in London last week, and here is the brief report.

There were 3 very useful talks (and a very good lunch after:) – as expected of Oracle I guess.

The conference was aimed at large organizations, with considerable amounts of sensitive data stored in Oracle databases and the way in which they store and transport the data.

i) Oracle provides pretty useful tool for encrypting all data in a database transparently, so applications and users still see data as normal, but the data is encrypted internally. It is the question of just turning it on and selecting the encryption algorithm for it to be used (from Enterprise Manager Web Interface) and the actual data stored in data files on hard disk drive will be encrypted on the actual medium.
With encryption turned on, if anyone gets hands on the hard drive from database server, or if someone nicks the entire server from the data centre (not unheard-of :) ) for example, actual data will be scribbled and so of no use to the intruder.

ii) Another utility shown provides a way to encrypt the database backups and exports as well, which can then be transported without fear of losing it (even if someone leave the DVD on the train or it gets lost in the post), the data will actually be encrypted, therefore unreadable, before it reaches the destination. The Oracle 10g+ Data Pump utility for data import/export support this out of the box.

iii) Finally, a couple of very useful tools for data masking. Imagine a complex application that deals with large amounts of data. In order for it to be developed as scalable, and to perform good, without bugs, the development and testing teams would require the real world data for the development process. However, since the data contains sensitive information, it cannot be given to them in its raw form. What development teams then resort to is the local database, with imaginary data. However, development or testing databases populated in this way usually don’t have the required amount of data (which may be in millions on the production system). What would be better is to use the actual production data, but with the sensitive columns masked, so they are unreadable, and therefore unavailable for any misuse while held at development and testing machines.
The demo shown used just a couple of mouse clicks to mask all sensitive data columns (names, email addresses, NI numbers…) – so ‘John Smith’ becomes ‘jhkjh laskjlkjlk’ for example. There are advance features as well, so you can mask sensitive columns but keep them human-readable, by using a set of rules or even an another set of data – so you can mask ‘John Smith’ to ‘Peter Taylor’ in the masked database. The masking process can be deterministic or not, depending on the security constraints. The deterministic approach is a bit less secure, but the data will always be masked in the same way, no matter how many times the masking process is run (therefore ‘John Smith’ from production database will always be masked as ‘Peter Taylor’ – so the testing teams can pick the ‘peter Taylor’ record every time, and each time be sure they are using the same database row as before). The non-deterministic approach will mask the data differently every time the masking is applied.

Each of these looks very useful and easy to use (on the demo at least:)). The demos shown used an Oracle Enterprise Manager web application to do the tasks, from the browser, but it is possible to run each of the from the command line or sqlplus as well.

Further resources:
http://www.oracle.com/technology/deploy/security/database-security/index.html
http://www.oracle.com/technology/products/oem/pdf/ds_datamasking.pdf
http://www.oracle.com/technology/deploy/security/database-security/transparent-data-encryption/index.html