Debugging Threads

This section assumes that you have a thread-capable gdbserver, like the one included in the Axis software distribution.

Note: if you get the message

  Ignoring packet error, continuing...
  Ignoring packet error, continuing...
  Reply contains invalid hex digit 79
  Stopped due to shared library event

you have to increase the remotetimeout in gdb (default is 2 seconds):

  set remotetimeout 60


The gdbserver uses libthread_db, a helper library, when debugging threads. If you are on a glibc-based system, libthread_db is already available. If you are on an uclibc-based system, make sure PTHREADS_DEBUG_SUPPORT is set in your .config file (or else gdbserver won't compile).

If the libthread_db library (or any other shared library) isn't available on your target you can upload it using e.g. FTP. In this case you must let the target system know where to look for the uploaded libraries. Set the environment variable LD_LIBRARY_PATH to point to where the uploaded libraries are stored.

If you upload the library to /mnt/flash/

# export LD_LIBRARY_PATH=/mnt/flash/

Pthread Debugging Example

The following is an example of what a pthread debugging session might look like, along with some basic commands. The example program below just creates 5 threads and waits for them to terminate. The call to sleep is just there so we'll have a chance at looking at the threads before they terminate.

  #include <pthread.h>
  #include <stdio.h>
  #define NUM_THREADS 5
  void *thread_function(void *arg)
     printf("Hello World from %d!\n", arg);
  int main(int argc, char *argv[])
     pthread_t threads[NUM_THREADS];
     int i;
     for (i = 0; i < NUM_THREADS; i++)
       pthread_create(&threads[i], NULL, thread_function, (void *)i);
     for (i = 0; i < NUM_THREADS; i++)
        pthread_join(threads[i], NULL);
     return 0;

First we set everything up and connect:

  (gdb) set solib-search-path /usr/local/cris/r63v32/crisv32-axis-linux-gnu/lib/
  (gdb) set solib-absolute-prefix /dev/null/
  (gdb) set remotetimeout 60
  (gdb) target remote
  Remote debugging using
  0x35557608 in ?? ()

Next we set a breakpoint in main after all threads have been created:

  (gdb) break hello.c:20
  Breakpoint 1 at 0x80568: file hello.c, line 20.

Now send the program on its way:

  (gdb) c
  [New Thread 1024]
  [Switching to Thread 1024]
  Breakpoint 1, main (argc=1, argv=0x9ffffe84) at hello.c:20
  20         for (i = 0; i < NUM_THREADS; i++)

The 'info threads' command displays information about the active threads:

  (gdb) info threads
    7 Thread 5126  0x355f04fe in nanosleep ()
     from /usr/local/cris/r63v32/crisv32-axis-linux-gnu/lib/
    6 Thread 4101  0x355f04fe in nanosleep ()
     from /usr/local/cris/r63v32/crisv32-axis-linux-gnu/lib/
    5 Thread 3076  0x355f04fe in nanosleep ()
     from /usr/local/cris/r63v32/crisv32-axis-linux-gnu/lib/
    4 Thread 2051  0x355f04fe in nanosleep ()
     from /usr/local/cris/r63v32/crisv32-axis-linux-gnu/lib/
    3 Thread 1026  0x355f04fe in nanosleep ()
     from /usr/local/cris/r63v32/crisv32-axis-linux-gnu/lib/
    2 Thread 2049  0x35608072 in __poll (fds=0xfffffffc, nfds=1, timeout=2000)
      at ../sysdeps/unix/sysv/linux/poll.c:63
  * 1 Thread 1024  main (argc=1, argv=0x9ffffe84) at hello.c:22

Notice that there are 7 threads, and not just the 5 that we created in the program. Besides the main thread there is also a management thread. Now switch thread using the 'thread' command:

  (gdb) thread 4
  [Switching to thread 4 (Thread 2051)]#0  0x355f04fe in nanosleep ()
     from /usr/local/cris/r63v32/crisv32-axis-linux-gnu/lib/

As usual, use 'backtrace' to find out how the thread ended up in nanosleep:

  (gdb) backtrace
  #0  0x355f04fe in nanosleep ()
     from /usr/local/cris/r63v32/crisv32-linux-gnu/lib/
  #1  0x355f0496 in __sleep (seconds=4294967292)
      at ../sysdeps/unix/sysv/linux/sleep.c:81
  #2  0x0008050c in thread_function (arg=0x1) at hello.c:8
  #3  0x3556e06a in pthread_start_thread (arg=0xfffffffc) at manager.c:262
  #4  0x3556e0b2 in pthread_start_thread_event (arg=0x9f5ffc00) at manager.c:285
  #5  0x3560d244 in clone ()
     from /usr/local/cris/r63v32/crisv32-axis-linux-gnu/lib/

Note. Sometimes you get an “corrupt stack?” message at the end of the call stack. Don't be alarmed by this message. It is explained in the Known problems section.

axis/gdb-debug-threads.txt · Last modified: 2007/03/08 12:21 by jesper
All text is available under the terms of the GNU Free Documentation License (see Copyrights for details).