Saturday, June 4, 2011

Development with pre-requisites and source code security.

Admittedly running a risk of combining two different subjects but they are so linked. One is development with pre-requisite, compiled libraries and other, source code security.
I want to write something on some of the problems arising out of having an extra large codebase, large and geographically scattered development teams. Certainly, we are looking at a distributed version control to help us out here. But I want to see what an individual developer needs to do to accomplish the daily bread-and-butter i.e to read and develop programs, build and test. This is particularly about individual source code management, ease of development and source code security.

Simple things first, how does a developer working on a small piece of that monstrous 4 million lines of code actually build and run the application on a measly desktop ? The actual run-time size of the application is much less troublesome as the time to download the code every morning because I have no idea which other developer has modified what other parts of the code and how it impacts my own code. Basically, there could be so many scattered changes coming in daily over the entire spectrum that the entire codebase on my desktop needs a refresh. Things go out of hand when you now consider that each developer will compile, build the entire code for testing any part of the application. The daily build time for each developer is the biggest blow here.

But really, each developer should just download and build code that is being touched by his/her team and now have to worry about all other code and treat it like a black-box. The UI development team almost never touches any code in Application Framework. There is a clear separation of concerns among those development teams. For testing, one needs to entire set of executables.

I have encountered this situation in my prevous stint working as a CAD-PLM application developer, for code written in C++. The method employed there is what I want to share.
We need a really sturdy server with a shared (and read-only) directory, that has a complete and "best-so-far" version of executables. A build machine does a daily build of the entire source code that is checked-in by the EOD and places all executables in this shared directory. We then have all developers map/mount this shared directory to some local directory.
Given this, the developer only downloads code that his/her team deals with and provides the shared directory as a 'include' and 'library' path after the local code. The compilers as well as the runtime environments happily work with the source code on local machine and pre-requisites on the shared path.

An intentional side-effect of this is that not all developers can see all source code, although they have all the executables to play with. The source is neatly divided into sub-directories containing code that belongs to a logical functionality. The version control system can explicitly disallow all but some users from checking-out certain such directories.
Why is hiding source sometimes desirable ,in today's computing world ? I can easily imagine a piece of source code that is truly break-through, with patented algorithms, technologies and what not. Then it follows that the organization does not want just "any" developer to get his hands on the source code - for securing it against tampering as well as source-code theft. What else but source code, is the most important asset of a software development company ? We have had several cases so far, where a sacked, disgruntled employee went out to sell his company's intellectual property for cool bucks. See reports here and here. After such incidents, those companies must have gone overboard, in searching employee baggage on each transit, for any smuggled electronic media.

All this might seem banal but if things do not get arranged in this manner to begin with, it is difficult to pull it back clean.

In the next few entries, I will explore a software licensing mechanism to control software usage, that will also hinge on some source code security. It all depends on some parts of the source code not bing readable by anyone. Secured source code is such a good thing.