CRiSP Weblog

I've added blowfish support to CRiSP - mainly because I wanted to (add some basic encryption facilities - reasons are too boring to go into). Each time I added encryption, its been a nuisance - no matter whether its my own coding or open source offerings.

Its important to ensure the code compiles identically across all platforms - any compiler bugs, language undefined behavior, or word sizes - can mean the difference between a file or block not being decryptable or checksummable on another platform. This behavior could go unnoticed for years. E.g. I used to do all my development on Solaris; after a number of years, I switched to Linux. Linux radically changes over the course of the years. And it would be "not nice" to find backups of files unreadable due to such a difference in behavior.

Even high quality open source code can blow up (silently) due to bugs in compilers or 32bit vs 64bit differences. Fortunately, most modern code is aware of these things, although many RFCs dont understand the real world (they provide sample algorithms, which may be proven, years later, to have buggy example implementations).

Good code comes with self-sanity checking (eg encrypt a block and decrypt should return the right/same answer). But there is little intermediate checking. Blowfish does this. Heres some sample code:

  Blowfish_Init (ctx, (unsigned char*)"TESTKEY", 7);
  Blowfish_Encrypt(ctx, &L, &R);
  if ((L & 0xffffffffU) != 0xDF333FD2L || (R & 0xffffffffU) != 0x30A71BB4L) {
  	printf("blowfish_encrypt: L=%lx R=%lx\n", L, R);
	printf("wanted: 0xDF333FD2L 0x30A71BB4L\n");
    	return (-1);
  }

Now, when the assertion fails - tracking down the lines of code which caused the issue is painful - every line of implementation code will "look" correct, but subtleties in sign-extension or zero filling or other ISO/C behavior can make obvious code never show the true issue.

All code should ideally have complete test cases, but you dont really know what to test for, until much much later, and by then, you may have even forgotten how it (or your own) code works.

I've recently been fixing annoyances in CRiSP and adding small/minor features.

One issue I was recently tracking down was a memory leak in the MacOS port. MacOS and the development environment is really nice. Using the GUI (X-Code) is too heavyweight on my frail Mac, but the command line tools are good. ("leaks" is a very nice tool). I spent a lot of effort tracking down a leak and fixing a piece of code, only to find that in trying to find a memory leak, involves the Mac tools *causing* a memory leak. (Effectively no memory is freed when one of the malloc debug options are turned on; took me ages to realise I had fixed my leak, but whilst monitoring for leaks, just showed process size growing as malloc was trying to detect leaks. Sometimes, its the obvious things).

Whilst on the subject of tools, I finally took the plunge and tried out Clang (2.9). Its a nice tool for static code analysis - and managed to uncover a few latent bugs in CRiSP, that I never knew existed. Sometimes I do like compilers with new features which are helpful. (Many gcc compiler warnings just generate noise, as does Visual Studio).

Meanwhile...back to figuring out why blowfish is not doing what is expected with my bits....

[Will resume dtrace shortly - I need to forget how it works, before tackling the next issue on the table - pid provider].