CRiSP Weblog

In the beginning was COFF - the common object file format. It was a great step up from prior binary file formats, since it had a structure and allowed enhancements to what was embodied in an executable or object file.

COFF was replaced with ELF - overcoming some of the limitations and hardcodings of COFF. It has been very successful. (Microsoft stuck with COFF; Unixes didnt).

ELF has a set of tables and sizes and allows arbitrary things in a binary such as debug symbols, relocatable symbols, and so on.

When it came to debugging, the initial attempts at containing symbol tables extended from symbol name/value pairs to structured descriptions of what an application does - no longer are name/value pairs enough to describe arrays-of-structs or class-members etc. This is inside one of the ELF sections, and uses a binary format called DWARF.

Most of this technology comes from Sun / Solaris (ELF, DWARF). DWARF was ported to other operating systems, and gcc/gdb and Linuxes happily support this.

Then libdw was created - with enhancements to libdwarf. Alas, documentation on both libraries is poor at best, and your eyes start to get very confused trying to figure out what the subtle differences are.

DTrace relies on libdwarf support for the CTF tools. When people build dtrace on a system without libdwarf, they see a diagnostic saying that ctfdump/ctfconvert are not available, and the build proceeds anyway.

CTF is the mechanism for taking the dwarf debug symbols from the kernel and making them available in DTrace D scripts, so you can refer to structures and members. When that is not present, it doesnt matter if all you want is to use the standard FBT and SYSCALL providers. It does matter if you want to do some other advanced things.

Ubuntu seems to support libdwarf but not libdw. Other Linuxes support libdw and not libdwarf. (Its difficult to tell if this is the truth, since my installations may be polluted by whatever I did a few years ago).

Its annoying today, eg when I install archlinux, that libdwarf is not supported but libdw is. So, trying to modify the code in cmd/ctfconvert to compile either way requires being an expert in both.

I may try over the next few days to see if I can fix this. With DWARF4 becoming more popular with later gcc or clang releases, this becomes more important to step up to the latest standards and tools.