building Python with the Boehm-Demers Garbage Collector

Introduction

Problem: Complex long-running Python systems that gobble up virtual memory.
Solution 1: Find all your cycles and remove them manually.
Solution 2: Plug in a garbage collector.

The Boehm-Demers-Weiser garbage collector from Xerox:
ftp.parc.xerox.com:/pub/gc/gc.tar.gz

Two approaches are described. The first makes almost no changes to Python; but uses the GC version of malloc and free. [reference counting is left intact] The second approach removes the reference-counting code, and requires patches to about 10 of the source files. This is a more dangerous option, but should perform better.

Building the GC library

If you're on linux, version 4.10 of the gc library may not build; a manual include of 'sigcontext.h' will do the trick. Near the top of "gc/os_dep.c":

#if 0
#   define __KERNEL__
#   include <asm/signal.h>
#   undef __KERNEL__
#endif
#   include <sigcontext.h>
# endif
# if !defined(OS2) && !defined(PCR) && !defined(AMIGA) && !defined(MACOS)
#   include <sys/types.h>
# endif
# include <stdio.h>
# include <signal.h>

Building Python

Stick this in Include/mymalloc.h just before the definition of PyMem_NEW:

#include "/usr/src/gc/gc.h"

#define malloc(n) GC_malloc(n)
#define calloc(m,n) GC_malloc((m)*(n))
#define realloc(o,n) GC_realloc(o,n)
#define free(n) GC_free(n)

The explicit include path is important! Don't use "-I/usr/src/gc", because the gc library also uses a 'config.h' file, which you don't want to include by mistake.
run configure.
Now, run 'make', but include the gc library in the definition of the symbol 'LIBS':

make LIBS="-lieee -ldl /usr/src/gc/gc.a"
           ^^^^^^^^^^^
           [these are taken from the LIBS definition in a normal Makefile
            yours may be different]


$ mv python gc-python

That should do it!

Building the no-refcnt version

This is a patched version of Python-1.5.1 that removes reference counting altogether. Start with a virgin copy of 1.5.1.

Grab python-gc-no-refcnt.patch

$ cd Python-1.5.1
$ patch --strip=1 < ../python-gc-no-refcnt.patch
$ ./configure --with-libs="/usr/src/gc/gc.a"
$ make OPT="-O -DPy_USE_GC"