CRiSP Weblog

I am planning a CRiSP v10 release imminently. My hobby project for Xmas has been to add pivot table support to the table widget.

CRiSP contains facilities for viewing CSV type files - but now its time to ramp up the facility a bit. The feature in Excel which allows you to do pivotal views - is nicely done.

What is a Pivot? In Unix terminology, a pivot is typically done via a series of grep commands piped into grep commands. This works to find lines matching or not matching a criteria, but isnt ideal for selecting columns:

$ grep pattern1 file | grep -v pattern2 | grep pattern3

Once you start to take columns into account, the above command line gets more tricky, and "the optimal way" to do it, gets muddy. E.g. you can use the cut(1) command, or awk, or perl.

CRiSP contains the powerful g// command (at the Command: prompt, use g/pattern/ to show all lines matching a pattern). Once armed with this list, you can further refine by filtering (<F>) to include or exclude certain lines.

Recent releases of CRiSP have included the command prompt 'col' command (type "col help" to see the commands available).

But the next step is to provide an Excel-like view (which already exists in the Find/Data-Miner edit option (this needs to be renamed and moved to the View or Tools menu).

The next evolution of this facility is a dropdown on each column, which shows the unique values. This, in itself is useful and interesting, but, with the addition of being able to enable or disable rows which match the column values, means, you get a simple SQL like facility for drilling down and summarising data.

Lets see how it evolves, before making it visible in the finished product.

I was browsing my mailbox this morning, as you normally do, and found a bunch of phishing emails. Nothing extraordinary. Its good that the phishers are people of very low IQ - makes you really wander how they cope with such complex difficulties as breathing or eating.

Anyway, in the spam is a bogus link to a banking website.

I am wondering, why we dont just bleed them dry. I know some people take a vigilante crusade on these people, but, if we all put in motion a "voting" scheme to vote down websites, e.g. a plugin to the browser/mailreader then you could just cause a mass blockage to a website by "clicking" on a link.

Maybe even the web browser should consult a white/black/dont-know list, and unless its been certified, it blocks access for you, on your behalf.

It could even look at the registration details of the site.

I am probably over simplifying the kinds of tactics one can do, or, have been done by people using honeypots etc.

I have been meaning to look into strace for a while - to see what causes = this message to be printed:

$ strace <application>
[ Process PID=3D17199 runs in 32 bit mode. ]
...

Interestingly, the code which prints this message determines the 32/64-bitness of an app by looking at the *instruction which invoked the syscall*. Heres the code:

switch (call & 0xffff) {
        /* x86-64: syscall = 0x0f 0x05 */
        case 0x050f: currpers = 0; break;

        /* i386: int 0x80 = 0xcd 0x80 */
        case 0x80cd: currpers = 1; break;

        default:
                currpers = current_personality;
                fprintf(stderr,
                        "Unknown syscall opcode (0x%04X) while "
                        "detecting personality of process "
                        "PID=%d\n", (int)call, pid);
                break;
}

What does this mean? Well, it agrees with my prior findings in dtrace, that on Linux, determining 32 or 64-bit ness is difficult because the kernel does not contain such an attribute. Strace is having to guess, and because it guesses, it can be lied to.

I havent tried this, but one could do an INT 0x80 from a 64-bit process (either we crash the kernel, or we succeed) but this would confuse strace. We could arrange to overwrite the instruction of the syscall (eg from another app or thread), and this too could cause strace to be defeated.

(You can make an application totally untracable, by forking your own tracer, which would preclude ptrace from another process from working).

In reality, the kernel knows what a binary type is via the 'personality' - effectively an object vmethod which is setup by virtue of execing the executable. (Shared libs are a problem, since they are just blobs of bytes, setup via an mmap arrangement, and could be anything - x86 32/64 bit, PPC or ARM, etc). So, an app can flip from 32 bit to 64 bit and back again, solely by some twiddling of what the PC points to.

(I am probably wrong on the latter points; you cannot suddenly flip the CPU from 32 to 64 and back again, without a protection level transition; but, a 64-bit app can execute a 32-bit instruction. Maybe I will have to try out this thought experiment and see what happens).

I see email piled up in my inbox. Like real mail, you just know that underneath the pile of spam and 419 scams, that there are real bills to pay...urgent email that just goes to the back of the queue.

I have a pile of 'Dtrace wont compile' emails in my inbox. At least one of them looks to be due to the fact the sender hasnt got the headers installed on his system .. an enhancement that is needed to the build script.

Heres one email, which just surprised me and made my day: partially because of the kind complement, but moreso, because I have *nothing* to do or follow up on:

Thank you so very much for your DTrace porting efforts..

Its pretty darn awesome. :)

(from a gentleman called Dan, somewhere in Denmark I presume).

I dont mind being wrong, sometimes .... :-)