../ Debug gblic in Arch Linux

Our ability as programmers is limited by the tools we use. Tools are extensions of our bodies. They let us experience life in a new and different way.

In the programming of systems, a foundamental ability is to be able debug a process while it is executing. To develop this ability we must have the right toolset at our disposal.

In this blog post I want to show how to debug gblic, the GNU implementation of the standard C library, using debuginfod in order to obtain debug source files and symbols. This method works also for debugging other libraries, if they are supported by debuginfod. In general the setup is simple and effective.

#Docker Environment

To make this guide reproducible I will use a basic docker container with just a base image of archlinux and an example of C code to debug.

The C code is shown below, written in a file named lcg.c. It uses functions taken from glibc, the GNU implementation of the standard C library. These functions are printf, rand, srand, initstate and setstate.

#include <stdio.h>
#include <stdlib.h>

int main(void) {
  char state1[8];
  initstate(1337, state1, 0);
  setstate(state1);

  srand(42);
  int n = 10;
  for (int i = 0; i < n; i++) {
    printf("%d\n", rand());
  }

  return 0;
}

The Dockerfile is shown below. We copy the source code we want to debug, and then we install basic packages to build and debug the code.

FROM archlinux:latest

COPY lcg.c /lcg.c

RUN pacman -Syy --noconfirm
RUN pacman -S which make gcc gdb icu wget --noconfirm

We can build the docker image as follows

$ docker build -t glibc-debug .

Sending build context to Docker daemon  20.99kB
Step 1/4 : FROM archlinux:latest
 ---> bca64474746f
Step 2/4 : COPY lcg.c /lcg.c
 ---> 2ec044bb6730
Step 3/4 : RUN pacman -Syy --noconfirm
...
Removing intermediate container 727cc2933476
 ---> 0c9d057ee548
Successfully built 0c9d057ee548
Successfully tagged glibc-debug:latest

And then we can create the container with the following command.

$ docker run --name glibc-debug --rm -t -d glibc-debug

To spawn a shell within the docker you can execute the following command.

$ docker exec -it glibc-debug "/bin/sh"

Once inside, the first thing we do is compile the binary with debugging symbols.

sh-5.2# cd /tmp
sh-5.2# ls
lcg.c
sh-5.2# gcc -ggdb lcg.c -o lcg

We can execute it as follows.

sh-5.2# ./lcg
71876166
708592740
1483128881
907283241
442951012
537146758
1366999021
1854614940
647800535
53523743

#Debuginfod

The simplest method to retrive the debugging information is by using debuginfod, which is a service used for providing debug information over an HTTP API. For more information, as always, read the related archwiki page.

https://wiki.archlinux.org/title/Debuginfod

To use this functionality we need to export the environment variable called DEBUGINFOD_URLS so that it will be used by gdb. This can be done as follows.

export DEBUGINFOD_URLS="https://debuginfod.archlinux.org/"

Once we have exported it, we can launch gdb with and start debugging. We put a breakpoint into rand and we send the run command to start execution. At this point the debugger will ask if we want to enable debuginfod for this session. We reply with y. Once we do that, it will proceed to download the source file on which we place the breakpoint, which in this case is glibc/stdlib/rand.c.

sh-5.2# gdb -q ./lcg
Reading symbols from ./lcg...
(gdb) b rand
Breakpoint 1 at 0x1080
(gdb) run
Starting program: /tmp/lcg
warning: Error disabling address space randomization: Operation not permitted

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.archlinux.org/>
Enable debuginfod for this session? (y or [n]) y
Debuginfod has been enabled.
To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit.
[Thread debugging using libthread_db enabled]                                                           Using host libthread_db library "/usr/lib/libthread_db.so.1".
Downloading source file /usr/src/debug/glibc/glibc/stdlib/rand.c
                                                                                                        Breakpoint 1, rand () at rand.c:26
26      {

At this point (or also at the beginning) we can execute tui enable to get the source code listing.

If you don't want to press y everytime, you can directly set within the .gdbinit config file the following line

set debuginfod enabled on

Debugging with this new addition of debuginfod means that we do not have to do a full compile build just to get the debug symbols and source code.

That's it!

#References

To finish off, some references regarding the topics covered.

Related to debuginfod

On glibc debugging in ubuntu

Others