Christmas PC Monday, 27 December 2010  
How to spend Xmas: buy a new PC. Get it working, and spend the holiday period fuming and swearing.

Ok, new PC (DELL - for what its worth - nice machine, good spec). It worked flawlessy.

So, first, we need to transfer photos and firefox bookmarks in order to keep the family happy. (PC is not for me).

So, we move out the old PC. Spend ages trying to get WinXP network and Windows 7 network to acknowledge each other. Eventually this works. On the way, I came across a very neat tool I hadnt used before.

At first, sharing the C drive wouldnt let me access the "\Documents and Settings" folders for the users. So, after a lot of googlework, CACLS.EXE came to the rescue. Alas, after a little driving error, I had managed to lock myself out of the folders (even as an admin). I grabbed a copy of TAKEOWN.EXE eventually, (having to transfer via USB drive and not the network - the network on the new PC became flaky for some reason...)

To cut a long story short, the files were transferred and PC duly handed over.

But the network was a problem: the old PC had a Linksys WMP54G PCI wireless card, but there are no drivers for Windows 7. So this was out of the question. So I dragged out some old Zyxel PLA401 homeplug blocks and this seemed to work. For a while. But one of the (4) devices I had kept losing the lights and was knackered.

So, I then tried using an Airport Express (which doesnt work on the mac, for some reason). But I needed to install the airport express utility - via USB from another machine, and this worked. But then I kept hitting the same problem as before - machine could see the device but never update it. Eventually, the airport express just refused to even see the device. (Thats 4 homeplus + 1 airport express that are useless).

So I grabbed out my old netgear wireless router - and that worked, except it had the same IP as the main router, so, after duly changing its IP address, I had more or less bricked it - I could no longer talk to the netgear - it didnt respond to the IP I thought it had and didnt respond to the factory default IP either.

So, back to the Zyxels - which worked for a while. (Did I mention that the Zyxel setup app wont install on a Windows 7 machine?) Tried using the setup app on the old PC, but this just showed that 3 out of the 4 blocks were bust.

After a lot more frustration, gave up on all of that kit and went out and bought a Belkin N300 USB drive. Success! It just worked.

So, now the home pc is signed off and I can forget this Xmas for many years to come.

But I do worry: any kit you buy today, may be useless in future operating systems - even finding the setup CDs can be painful, but so many "bits of hand waving" magic are needed to get things to work.

And Linux is no better in this regard: (recall, my software suspend doesnt work, and I need a.n.other PhD in Linux kernel changes to see why 2.6.32 kernel works but 2.6.35 and 2.6.36 does not).

Oh to do some more coding where external dependencies dont get in my way.

Posted at 19:32:44 by fox | Permalink
  elfrewrite - why did i bother? Monday, 27 December 2010  
Someone kindly suggested that the effort in building elfrewrite was wasted and that there are many well-known existing methods to achieve the result of a universal binary.

I acknowledge there are many ways to do this, but none of them fulfil my need or desire.

In days of old, one would pick an old release of a platform and stick with it. I stuck with HPUX-10 for more than a decade, AIX3.2 for more than a decade, and for Solaris, 2.6 was a good platform.

But Linux is different. Each new release of kernel or glibc was like a "bomb" - you open it carefully, and see what surprise are in store. glibc introduces many versioning dependencies as key libc functions are enhanced in a way that is not backwards compatible.

A suggestion - and one I use religiously, is Virtual machines - have your build environment in a VM. I do this, but I dont like building in a VM something different from the main platform I use daily. I currently use Ubuntu 10.10 and will upgrade to 11.04 when that comes out. I have issues with Ubuntu, but it suffices and most of my hardware works (apart from software suspend).

elfrewrite has no dependencies (other than itself and a libelf library written from scratch, as part of the CRiSP build environment).

What elfrewrite allows me to do, is, whenever an app fails to run on another OS platform (AS4, AS5, archlinux, etc), I can run it on the target binary and be done. No rebuild from source; no need to determine which version of gcc/ld had -Wl,hash-style= implemented, and a possibility to enhance the mechanism to allow performance or "working" or both.

Why do I even bother to write and publicise this?

Because I can. Because it interests me. Because when I first hit the problems leading to its necessity, determining the root cause was impossible even with google. Now I know what to search for and understand the rationale behind these changes, and there are many good articles on the web on the innards of ELF. But there are far more "why doesnt X run on my system?" along with various non-useful suggestions (involving upgrading parts of a system you may have no control over, to rebuilding an app). I gave up building FSF apps a long time ago, when faced with the complexity of lib dependency and broken autoconfig make systems.

We all write code to build something better, and we dont have time to decipher every rationale and change in a foreign piece of code - most open source apps have become too complicated (and I dont mean that in a negative sense, but simply trying to make enhancements or fixes is difficult because of the size of the apps).

Posted at 19:19:27 by fox | Permalink
  Xmas and elfrewrite is finished Saturday, 25 December 2010  
Took longer than I had hoped/expected - the base code/algorithm got rewritten about 6 times, as each code scenario played out, had a crink - me taking a shortcut.

The most recent prior release had issues because I had fudged a "hole" into the ELF binary and although the file was well formed, libbfd didnt like this (eg "strip" gave all sorts of diagnostics and wrote out an invalid ELF file as a result).

The final algorithm is simple: add the .hash table just prior to the start of the LOAD/phdr code segment. Rather than appending, we are prepending. We have a nice free chunk of address space to achieve this, and the first attempt was blighted by smashing in the data without honoring the contract that the PHDR should following the EHDR.

Here is what an ELF file should look like:

| Elf Ehdr      	     |
| PHDR (Program Hdr)         |
| section #1 data (.text)    |<--- 
|			     |	  |
|			     |	  |
|----------------------------|	  |
| section #2 data (.data)    |	  |
|			     |<--------
|			     |	  |   |
|----------------------------|	  |   |
|      ....     	     |	  |   |
|			     |	  |   |
|			     |	  |   |
|			     |	  |   |
|----------------------------|	  |   |
| Section table 	     |	  |   |
|    | #1  | addr | size |   |----    |
|    | #2  | addr | size |   |--------
|    | ... |      |      |   |

The program header is important because it is used by the kernel to load the needed segments into memory (via mmap() type technology) and most apps will require two memory map ranges - one for the .text/code and another for initialised data. A real executable typically contains 30 or more sections, breaking down the app into code, read-only data, bss data, and various housekeeping segments for things like debugging, and relocations.

The format itself is elegant and simple, and extensible. The libelf libraries provide object-oriented like access to such files, for reading or creating, but the library can be opaque and difficult to use when things go wrong.

Posted at 09:57:59 by fox | Permalink
  ELF rewriting - working again Monday, 20 December 2010  
I had to throw away the algorithm in my last post to work properly for ELF32 files where the text and code do not provide a gap to stuff the new .hash in.

The new code is working much better for ELF32 and ELF64 - but I have a minor bug to fix ("strip"ping the executable results in damage, because I broke a rule somewhere). I am creating a spare area just before the .text section is loaded, and creating a dummy section to point to it, but libbfd (as used in "strip") doesnt like me; not surprising really, as I am being mean and nasty.

Posted at 23:21:24 by fox | Permalink
  ELFrewrite - update Sunday, 19 December 2010  
Darn. One of my target executables doesnt have enough space to patch in the new hash table, so the code I posted fails for the ELF32 scenario for at least one app.

Oh well...back to the drawing bored :-)

Posted at 22:43:51 by fox | Permalink
  ELF - elfrewrite finished Sunday, 19 December 2010  
After a hard battle, it looks like it works. I dont promise this will work for everyones executables, but it works for my Linux 32 and 64 bit crisp executables.

I had to totally take a different direction for ELF32. For ELF32, we dont have a gap between .text and the .data/.bss and other sections, so we cannot shoe horn in the new .hash table.

Instead, we can overwrite the .gnu.hash with a .hash conforming table - assuming our table is no larger than .gnu.hash. (.gnu.hash is supposed to be smaller than .hash, but looks like for ELF32 binaries, this is not the case).

Ideally, the next step is to package up the binary and/or source so others can play with it.

I wrote my own ELF library, rather than rely on -lelf and -lgelf, since I hit some horror stories in those libraries and didnt fancy debugging or coping with older and newer libraries.

My elf library tries to hide some of the 32/64 bit ELF differences, and is "not bad" - could do with more work to shield more differences, but the design pattern of handling ELF32 + ELF64 is fine.

I also had to enhance my very old elfdump tool (like readelf and objdump), since I was not happy with either of those: they work "mostly", but when tracking down ELF brokeness, they can fall over or just ignore the issue.

I attach the code below, which wont compile, because of the need for the libraries, but, if theres enough interest, I will make it available.

/*                                                                    */
/*      CRiSP - Programmable editor                                   */
/*      ===========================                                   */
/*                                                                    */
/*  File:          elfrewrite.c                                       */
/*  Author:        P. D. Fox                                          */
/*  Created:       16 May 2010                                        */
/*                                                                    */
/*  Copyright (c) 2010, Foxtrot Systems Ltd                           */
/*                All Rights Reserved.                                */
/*                                                                    */
/*  Description:  Tool to create portable Linux binaries              */
/*   $Header: Last edited: 16-May-2010 1.1 $			      */
/*                                                                    */
/*   ld --hash-style=both [sysv|gnu|both]			      */

/* gcc -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -o elfrewrite elfrewrite.c -lelf make make_foxlib ; gcc -g -I. -DCR_LINUX_X86_64 -Iinclude -Ifoxlib/elf -o /tmp/rw ~/tmp/rw.c bin/foxlib.a -lelf && cp /tmp/x1 /tmp/x && /tmp/rw /tmp/x */

# include <machine.h> # include <stdio.h> # include <string.h> # include <stdlib.h> # include <unistd.h> # include <libelf.h> # include <elf.h> # include <gelf.h> # include <sys/types.h> # include <sys/stat.h> # include <elf/elflib.h>

# define TRUE 1 # define FALSE 0

static int debug; static int v_flag;

char *fname; int patched = FALSE; int patched_strtab = FALSE; void *hash_addr; elf_t *elf;

/**********************************************************************/ /* Prototypes. */ /**********************************************************************/ int get_hash_size(int n); int do_file(void); void patch_dynamic(void); void patch_glibc_functions(void); void patch_hash(void);

int do_switches(int argc, char **argv) { int i;

for (i = 1; i < argc; i++) { char *cp = argv[i]; if (*cp++ != '-') return i; while (*cp) { switch (*cp++) { case 'd': debug = TRUE; break;

case 'v': v_flag = 1; break; } } } return i; } int main(int argc, char **argv) { int i, arg_index;

arg_index = do_switches(argc, argv);

if (arg_index >= argc) { printf("Usage: elfrewrite <a.out>\n"); exit(1); }

for (i = arg_index; i < argc; i++) { patched = FALSE; patched_strtab = FALSE; hash_addr = NULL; fname = argv[i]; do_file(); } exit(0); } int do_file() { # if !defined(HAVE_LIBELF_H) printf("Sorry, libelf.h not on this system - so this is a dummy\n"); return 0; # else

if ((elf = elf_read(fname)) == NULL) { char *cp; int ret = elf_get_error(&cp); printf("ELF error: %d %s\n", ret, cp); return -1; }

/***********************************************/ /* Patch GLIBC functions. */ /***********************************************/ patch_glibc_functions();

/***********************************************/ /* Patch the .gnu.hash or add a .hash */ /* hiding in the ELF. */ /***********************************************/ patch_hash();

/***********************************************/ /* Patch the .dynamic entries, e.g. based */ /* on the hash table updates. */ /***********************************************/ patch_dynamic();

/***********************************************/ /* Now save the results. */ /***********************************************/ if (patched) { if (elf_write_file(elf, fname) < 0) { printf("Failed to write file\n"); perror(fname); } }

elf_free(elf); return 0; #endif } /**********************************************************************/ /* Patch the entries which cause versioning issues or .gnu.hash */ /* issues on older glibcs. */ /**********************************************************************/ void patch_dynamic() { char *cp; char *cpend; char *ip; int dynamic; int size = elf->is_64 ? sizeof(long) : sizeof(int);

/***********************************************/ /* Patch the Dynamic section for GLIBC */ /* dependencies. */ /***********************************************/ if ((dynamic = elf_get_section_by_name(elf, ".dynamic")) < 0) return;

cp = elf->e_sections[dynamic].s_data; cpend = cp + elf->e_sections[dynamic].s_size; for (ip = cp; ip < cpend; ip += 2 * size) { long v = elf->is_64 ? ((long *) ip)[0] : ((int *) ip)[0]; long v1 = elf->is_64 ? ((long *) ip)[1] : ((int *) ip)[1]; switch (v) { case DT_VERNEED: # define padding 0x8000000 printf("DT_VERNEED: %08lx patched\n", v1); if (elf->is_64) ((long *)ip)[0] = padding; else ((int *)ip)[0] = padding; patched = 1; break;

case DT_VERNEEDNUM: printf("DT_VERNEEDNUM: %08lx patched\n", v1); if (elf->is_64) ((long *)ip)[0] = padding; else ((int *)ip)[0] = padding; patched = 1; break;

case DT_VERSYM: printf("DT_VERSYM: %08lx patched\n", v1); if (elf->is_64) ((long *)ip)[0] = padding; else ((int *)ip)[0] = padding; patched = 1; break;

case DT_GNU_HASH: printf("DT_GNU_HASH: %08lx -> %08lx\n", v1, hash_addr); if (elf->is_64) { ((long *)ip)[0] = DT_HASH; ((long *)ip)[1] = hash_addr; } else { ((int *)ip)[0] = DT_HASH; ((int *)ip)[1] = hash_addr; } patched = 1; break; } } } /**********************************************************************/ /* Patch the versionised glibc functions to be non-versionised. */ /**********************************************************************/ void patch_glibc_functions() { int i; char *cp; char *cpend;

for (i = 0; i < elf->e_shnum; i++) { if (elf->e_sections[i].s_type != SHT_STRTAB) continue;

cp = elf->e_sections[i].s_data; cpend = cp + elf->e_sections[i].s_size; if (debug) printf("Examining strings in %s\n", elf_section_name(elf, i)); while (cp < cpend) { char *next_cp = cp + strlen(cp) + 1; if (debug) printf("str=%s\n", cp); if (strcmp(cp, "__isoc99_sscanf@@GLIBC_2.7") == 0) { strcpy(cp, "sscanf"); patched = TRUE; patched_strtab = TRUE; printf("%s: patched %s: __isoc99_sscanf@@GLIBC_2.7 -> sscanf\n", fname, elf->e_sections[i].s_name ); } if (strcmp(cp, "__isoc99_sscanf") == 0) { strcpy(cp, "sscanf"); patched = TRUE; patched_strtab = TRUE; printf("%s: patched %s: __isoc99_sscanf -> sscanf\n", fname, elf->e_sections[i].s_name ); } cp = next_cp; } } } /**********************************************************************/ /* Patch the hash table in for older glibc implementations. */ /**********************************************************************/ void patch_hash() { int i; int hash; int gnu_hash; char *cp; char *cpend; int dynsym; int dynstr; int nsyms; int seg; int nchain, nbuckets, w_size; int first_chain; Elf64_Word *bucket_array; Elf64_Word *chain_array; Elf64_Word *wp;

/***********************************************/ /* See if we have a .hash or .gnu_hash */ /***********************************************/ hash = elf_get_section_by_name(elf, ".hash"); gnu_hash = elf_get_section_by_name(elf, ".gnu.hash"); if (hash > 0) return;

if (debug && hash < 0) { if (gnu_hash < 0) printf("Missing .hash and .gnu.hash sections\n"); else printf("Missing .hash, but have .gnu.hash section\n"); }

dynstr = elf_get_section_by_name(elf, ".dynstr"); dynsym = elf_get_section_by_name(elf, ".dynsym"); nsyms = elf->e_sections[dynsym].s_size / sizeof(Elf64_Sym);

/***********************************************/ /* Create the hash table. */ /***********************************************/ first_chain = 1; nchain = nsyms; nbuckets = get_hash_size(nchain); w_size = (1 + /* zero entry unused */ 1 + /* nbuckets */ 1 + /* nchain */ nbuckets + nchain) * sizeof(*wp); wp = calloc(w_size, 1);

wp[0] = nbuckets; wp[1] = nchain;

if (v_flag) printf("w_size=%d, buckets=%d, chains=%d\n", w_size, nbuckets, nchain); bucket_array = wp + 2; chain_array = bucket_array + nbuckets;

for (i = 1; i < nchain; i++) { Elf64_Sym *symp = (Elf64_Sym *) elf->e_sections[dynsym].s_data + i; char *name = elf->e_sections[dynstr].s_data + symp->st_name; int h = elf_hash((unsigned char *) name) % nbuckets; chain_array[first_chain] = bucket_array[h]; bucket_array[h] = first_chain; first_chain++; } //printf("crc=%lx\n", crc32(wp, w_size));

/***********************************************/ /* Merge this into the executable code */ /* segment. We have two strategies. For */ /* ELF32, we can overwrite the .gnu.hash */ /* section, since our hash is smaller than */ /* the generated one. (This isnt */ /* guaranteed, but gets us out of a hole). */ /* The hole is that in ELF32, the data+bss */ /* immediately follows the .text area and */ /* theres not a big enough gap to slide our */ /* new hash table into the binary. */ /* */ /* For ELF64, we tack the hash chain onto */ /* the last data section - since the */ /* .gnu.hash section is typically too small */ /* for us. (We could probably */ /* compress/optimise the hash table, but it */ /* is touch and go). */ /* */ /* We want a unified approach, but */ /* debugging and handling sections moving */ /* can cause us to need to do a full */ /* relink, which is even more treacherous. */ /***********************************************/

/***********************************************/ /* Handle ELF32. */ /***********************************************/ if (elf->is_64 == FALSE) { if (elf_section_size(elf, gnu_hash) < w_size) { printf("Help! Need %d section, but only have %d (.gnu.hash)\n", w_size, elf_section_size(elf, gnu_hash)); exit(1); } hash_addr = elf_section_addr(elf, gnu_hash); memcpy(elf->e_sections[gnu_hash].s_data, wp, w_size); return; }

/***********************************************/ /* Handle ELF64. */ /***********************************************/

/***********************************************/ /* Round up size to a 4K boundary, else */ /* kernel will KILL the binary due to */ /* misalignment of the data LOAD Phdr. */ /***********************************************/ int w_size1 = (w_size | (0x1000 -1)) + 1;

/***********************************************/ /* Find first writable section - we want */ /* the one before that, so we can dump our */ /* payload in. */ /***********************************************/ for (i = 1; i < elf->e_shnum; i++) { if (elf->e_sections[i].s_flags & SHF_WRITE) break; } i--;

if (debug || v_flag) printf("Extending seg#%d\n", i);

/***********************************************/ /* Modify the target segment to append our */ /* payload. */ /***********************************************/ cp = malloc(elf->e_sections[i].s_size + w_size); memcpy(cp, elf->e_sections[i].s_data, elf->e_sections[i].s_size); memcpy(cp + elf->e_sections[i].s_size, wp, w_size); w_size = w_size1; free(elf->e_sections[i].s_data); elf->e_sections[i].s_data = cp; elf->e_sections[i].s_size += w_size;

/***********************************************/ /* We need to update the .dynamic section */ /* to put our .hash "segment" in, so */ /* remember where we left it. */ /***********************************************/ if (elf->is_64) hash_addr = (void *) (elf->e_shdr64[i].sh_addr + elf->e_shdr64[i].sh_size); else hash_addr = (void *) (elf->e_shdr32[i].sh_addr + elf->e_shdr32[i].sh_size);

/***********************************************/ /* This is horrible but necessary - destroy */ /* the ".eh_frame" name so that the */ /* debugger cannot find it, otherwise we */ /* will core dump gdb when hitting a */ /* breakpoint. */ /***********************************************/ if (elf->is_64) elf->e_shdr64[i].sh_name++; else elf->e_shdr32[i].sh_name++;

/***********************************************/ /* Update subsequent sections because we */ /* moved them in memory (these only affect */ /* the loadable offset, not the p/v addr). */ /***********************************************/ if (elf->is_64) { elf->e_shdr64[i].sh_size += w_size; for (i++; i < elf->e_shnum; i++) { elf->e_shdr64[i].sh_offset += w_size; } } else { elf->e_shdr32[i].sh_size += w_size; for (i++; i < elf->e_shnum; i++) { elf->e_shdr32[i].sh_offset += w_size; } }

/***********************************************/ /* Now update the program header so the */ /* kernel can load the executable with the */ /* updated file layout. */ /***********************************************/ seg = elf_phdr_find_by_type(elf, PT_LOAD); if (elf->is_64) { Elf64_Phdr *p = (Elf64_Phdr *) elf_phdr_ptr(elf, 0); p[seg].p_memsz += w_size; p[seg].p_filesz += w_size; p[seg+1].p_offset += w_size; } else { Elf32_Phdr *p = (Elf32_Phdr *) elf_phdr_ptr(elf, 0); p[seg].p_memsz += w_size; p[seg].p_filesz += w_size; p[seg+1].p_offset += w_size; } } /**********************************************************************/ /* Compute .hash bucket size. */ /**********************************************************************/ int get_hash_size(int n) { int i; /**********************************************************************/ /* Table of bucket sizes depending on the symbol table size. */ /**********************************************************************/ # define NBKTS 2579 static const int hashsize[] = { 3, 17, 31, 37, 67, 97, 131, 197, 263, 397, 619, 1039, 1553, 1709, 1949, 2711, 4019, 7177, NBKTS };

for (i = 0; i < (int) (sizeof hashsize / sizeof hashsize[0]) - 1; i++) { if (hashsize[i+1] >= n) return hashsize[i]; } printf("Too many items to hash, please extend the array: %d\n", n); return NBKTS; }

Posted at 21:42:34 by fox | Permalink
  ELF Format files Saturday, 18 December 2010  
Its been an interesting couple of weeks trying to get the ELF converter to create a universal binary for Linux. With later GLIBC releases, the .hash section was replaced with a .gnu.hash section, and the /lib/ runtime linker understands this new section. (The new section is smaller and faster to link, especially with C++ apps which have long mangled names, and potentially hundreds or thousands of dynamic symbols, as is typical for GUI applications).

What I have uncovered is redundancy in the ELF specification. There is a Program Header section which is used to bootstrap the executable into memory. Heres an example program header:

Program headers:
   Type        Offset VirtAddr   MemSz/FileSz  PhysAddr     Flgs Al
0 PHDR         000040 00400040     1f8/1f8     000000400040 R-X  08
1 INTERP       000238 00400238      1c/1c      000000400238 --X  01
2 LOAD         000000 00400000    172c/172c    000000400000 R-X  200000
3 LOAD         001e28 00600e28     210/200     000000600e28 -WX  200000
4 DYNAMIC      001e50 00600e50     190/190     000000600e50 -WX  08
5 NOTE         000254 00400254      44/44      000000400254 --X  04
6 GNU_EH_FRAME 00068c 0040068c      24/24      00000040068c --X  04
7 GNU_STACK    000000 00000000       0/0       000000000000 -WX  08
8 GNU_RELRO    000e28 00600e28     1d8/1d8     000000600e28 --X  01

The LOAD sections are used to mmap the two key sections (code and data) into the memory image. Note the offset field: this specifies the byte offset into the ELF file where the segment resides. The runtime linker wants this to be on a page boundary.

The VirtAddr and PhysAddr control how the app can address these sections whilst it is running. (I dont know why both fields are present or what happens if they are different - I think one is redundant).

In addition to the Program headers, we have the section table. Think of the Program header as what the runtime linker needs to mmap the data into memory, and the section table as what the application uses internally, although much of the sections are irrelevant to a running program - the act of linker resolves addresses and so there is little need to use the sections. (The runtime linker will use some of the sections, e.g. to resolve dynamic symbols).

Then we have the DYNAMIC section (.dynamic) which is an array of values - some are pointers, some are integers, used to list things like shared lib dependencies or pointers to the hash table.

Section .dynamic: 0x1e50, size=0x190, num=25
[ 0]  00000001 DT_NEEDED     0x00000010
[ 1]  0000000c DT_INIT       0x00400428
[ 2]  0000000d DT_FINI       0x00400668
[ 3]  00000004 DT_HASH       0x0040072c
[ 4]  00000005 DT_STRTAB     0x00400330
[ 5]  00000006 DT_SYMTAB     0x004002b8
[ 6]  0000000a DT_STRSZ      0x00000057
[ 7]  0000000b DT_SYMENT     0x00000018
[ 8]  00000015 DT_DEBUG      0x00000000
[ 9]  00000003 DT_PLTGOT     0x00600fe8
[10]  00000002 DT_PLTRELSZ   0x00000048
[11]  00000014 DT_PLTREL     DT_RELA
[12]  00000017 DT_JMPREL     0x004003e0
[13]  00000007 DT_RELA       0x004003c8
[14]  00000008 DT_RELASZ     0x00000018
[15]  00000009 DT_RELAENT    0x00000018
[16]  08000000 134217728     0x00400398
[17]  08000000 134217728     0x00000001
[18]  08000000 134217728     0x00400388
[19]  00000000 DT_NULL       0x00000000
[20]  00000000 DT_NULL       0x00000000
[21]  00000000 DT_NULL       0x00000000
[22]  00000000 DT_NULL       0x00000000
[23]  00000000 DT_NULL       0x00000000
[24]  00000000 DT_NULL       0x00000000

So, we now have *three* places with similar information, and these can disagree. Here are some things to contemplate when they agree/disagree:

1. The kernel will refuse to load the executable of kill -9 on startup if the LOAD program header segments are not well formed (eg not on a 4K boundary).

2. The executable may run, but gdb may core dump. Older gdb's, e.g. 6.8, will core dump on a malformed executable, despite it running correctly.

3. gdb may run, but core dump on hitting a breakpoint. Even the gdb 7.2 release on Ubuntu 10.10 does this.

Combine the above with Ubuntu 10.10 vs Centos 4.7 (RedHat AS4) and the permutations are a bit unwieldy. Also, because the runtime linker on the earlier Linux's is using different fields, highlights the redundancy and problems in getting this to work.

So, at present I have an executable compiled on Ubuntu 10.10 which runs fine on Centos 4.7, but I now need to determine why the gdb on both systems is core dumping.

Posted at 21:57:10 by fox | Permalink
  0x000000006ffffef5 (GNU_HASH) 0x4005f0 Thursday, 02 December 2010  

This is the output from a 'readelf -a /bin/ls'. It is the cause of the dynamic linker problem (glibc 2.5 or later). By allowing /bin/ld write this into the ELF executable, the /lib/ does not like it on the older systems. Additionally, we are missing a ".hash" ELF section, but the above is a trigger.

I have a tool which removes the .gnu.hash section and adds a .hash section. Whilst the above .dynamic entry exists, it doesnt know what I did. Additionally, gdb gets confused too.

Next step is to polish off the code so it can create a fully conforming/working ELF executable, and its ready to roll.

(In case you didnt follow any of this, how to create an executable on Linux which runs on any release of Linux/glibc. Later Linux distros dont create backward compliant executables leading to an arms race of platform and version specific releases which is a waste of everyones' effort).

Posted at 20:40:37 by fox | Permalink