Chapter 31
Memory Debugging Utilities: liboskit_memdebug.a

31.1 Introduction

The Memory Debug Utilities Library is a set of functions which replace the standard OSKit memory allocation functions, see Section 14.5, of the minimal or FreeBSD C libraries. The replacement routines detect problems with memory allocation, and can print out file and line information, along with a back-trace to the offending allocation.

All of the standard functions are covered: malloc, memalign, calloc, realloc, free, and smalloc, smemalign, and sfree.

To use the library, just include -lmemdebug on the linker command line before the standard C library (or wherever it is the standard allocation routines are coming from).

The memdebug library implements a fence-post style malloc debug library. It detects the following problems:

Whenever a problem is encountered a back-trace (in the form of program counter values) is dumped (back-tracing from the allocation of the memory). File and line number information from where the allocation call was made are also printed (if available). If the failure was detected in a call to free, the file and line of that call are printed. This is called a “bogosity dump.”

When correctable errors are detected (e.g., sfree’ing a malloc’d block, or sfree’ing with the wrong size block). the correct thing will be done, and the program will continue as normal (except for the bogosity dump).

Note that file and line number information is only available if you’re using the macro wrappers for the allocators defined in memdebug.h. The call stack trace is always available.

One of the shortcomings of the library is that errors are only detected during explicit calls into the library, and not at the time that they happen. The memdebug_sweep function will check the validity of all allocated blocks, and by judiciously sprinkling calls throughout your code you can narrow down memory trashing problems. Similarly, the memdebug_ptrchk function will run a sanity check on a single pointer. Both functions, when printing “bogosity dumps” will also print the file and line at which they were called.

To help detect leaks of unfreed memory, use memdebug_mark and memdebug_check. memdebug_mark tags all allocated blocks, and then memdebug_check will check for untagged blocks. In this way, you can “mark” all blocks as okay and at a later point when all memory allocated after the mark should have been released, insert a “check”. The library will print a bogosity dump for any allocation that is untagged.

To help detect accesses after memory is released, or accesses to uninitialized memory, the library sets all bytes of an allocation to:

31.1.1 Memdebug Library Configuration

There are several configuration options in the library-private memdebug.h header file. The NO_MEM_FATAL #define controls whether errors in an allocation are fatal (via panic) or if they return 0. The #define ALLOW_MORALLY_QUESTIONABLE_PRACTICE controls the library’s handling of malloc(0) and free(NULL). While both of these constructs are technically legal, they usually signal errors in the caller; the option merely controls whether a message is printed or not. The MALLOC_0_RET_NULL option controls the behavior of malloc(0), either returning NULL or returning a valid, unique (per-allocation) pointer.

31.1.2 Memdebug Library Internals

The memdebug library uses two internal routines, memdebug_untraced_alloc and memdebug_untraced_free to actually allocate and free the memory it tracks. The default implementations of these routines use the initial system memory object (see Section 13.4). An implication of this is that the unwrapped malloc and the memdebug wrapped malloc can have different policies. This would be the case if the client OS has provided its own implementation of malloc not based on the system memory object.

When allocating memory on small alignment boundaries, those boundaries will actually be bumped up to the alignment necessary for the leading fence-post of the allocation. Thus, when running under memdebug data may be aligned at a larger granularity than when running without memdebug.

All of the routines use memdebug_printf to print all output. This function should always be defined such that it guarantees that it will never cause any memory to be allocated. You should override this if you cannot guarantee that vfprintf calls will not allocate memory.

31.1.3 External Dependencies

The memdebug library uses several functions, and one global variable that it does not define. It uses panic for flagging internal consistency failures, and memset for wiping swaths of memory. The default implementation of memdebug_printf requires vprintf.

For memory allocation primitives, the memdebug library depends on memdebug_untraced_alloc and memdebug_untraced_free. As mentioned, the default versions of these depend on the initial system memory object as provided by whatever C library is in use. Additionally, calls to mem_lock and mem_unlock are used to protect accesses to memdebug’s internal memory lists. These routines are described in more detail in the Memory Allocation section of the Minimal C Library chapter, Section 14.5.)

31.2 Debugging versions of standard routines

The functions listed below are defined as macros in the header file oskit/memdebug.h, they are also defined as simple wrappers in the library. The macro versions provide the library with file and line number information.

They are drop-in replacements for the allocation functions described in Section 14.5.

void *malloc(size_t size);
void *realloc(void *buf, size_t new_size);
void *calloc(size_t nelt, size_t elt_size);
void *memalign(size_t alignment, size_t size);
void free(void *buf);
void *smalloc(size_t size);
void *smemalign(size_t alignment, size_t size);
void sfree(void *buf, size_t size);

31.3 Additional Debugging Utilities

These routines provide additional features useful for tracking down memory leaks and dynamic memory corruption.

Mark all currently allocated blocks
Look for blocks allocated since mark that haven’t been freed
Check validity of a pointer’s fence-posts
Check validity of all allocated block’s fence-posts

These routines are internal to the memdebug library, but may be worth overriding in your system.

A standard printf-style routine that can be guaranteed to not allocate any memory.
Dumps information about an allocation block when an error in the block is detected.
Stores a back-trace (the call-stack) in a provided buffer.
Obtain memory from the client OS.
Return memory to the client OS.

31.3.1 memdebug_mark: Mark all currently allocated blocks.


#include <oskit/memdebug.h>

void memdebug_mark(void);


This function walks the list of all allocated objects and “marks” them. This is useful so that you can determine what was allocated before a certain point in your program.

Objects only have one bit to keep track of marks, so calling memdebug_mark more than once may not have the effect you would like.



31.3.2 memdebug_check: Look for blocks allocated since mark that haven’t been freed.


#include <oskit/memdebug.h>

void memdebug_check(void);


This functions walks the list of all allocated blocks and for each block that is not marked (by memdebug_mark), it prints a bogosity dump.

For example, at the beginning of a server loop call memdebug_mark, then when the server loop is about to iterate, call memdebug_check to make sure that the loop didn’t leave any allocated objects lying about.


memdebug_bogosity, memdebug_mark

31.3.3 memdebug_ptrchk: Check validity of a pointer’s fence-posts


#include <oskit/memdebug.h>

int memdebug_ptrcheck(void* ptr);


This function runs a host of sanity checks on a given pointer. Of course, these only work if the pointer, ptr is one returned by a memdebug-wrapped allocator. For any errors a bogosity dump is printed.

A pointer to a memory block allocated by some memdebug wrapped allocator.

Returns -1 if the fence posts are trashed so badly that the information in them cannot be trusted. Returns 1 if there was a problem detected but it is not “fatal”. Returns 0 if everything is A-okay.



31.3.4 memdebug_sweep: Check validity of all allocated block’s fence-posts


#include <oskit/memdebug.h>

void memdebug_sweep(void);


This function walks the list of all allocated blocks and calls memdebug_ptrchk on each entry.



31.3.5 memdebug_printf: A printf-style routine guaranteed not to allocate memory


#include <oskit/memdebug.h>

int memdebug_printf(const char *fmt, ...);


Works just like standard libc printf, but this function must be guaranteed to not allocate any memory while it runs.

The standard printf format string.
The standard printf arguments for the specific format string.

Returns the standard printf return value.

31.3.6 memdebug_bogosity: Prints a memdebug bogosity message


#include <oskit/memdebug.h>

void memdebug_bogosity(memdebug_mhead *head);


Prints a bogosity dump given the first fence-post of an allocation. Uses memdebug_printf for all output.

This routine is called by all others in the library to dump information about an allocation.

The head fence-post for the given allocation. Contains the back-trace, file and line number information, and allocation-style information.


31.3.7 memdebug_store_backtrace: Stores call-stack trace in provided buffer


#include <oskit/memdebug.h>

void memdebug_store_backtrace(unsigned *backtrace, int max_len);


Stores a machine-specific back-trace in the provided buffer. In conjunction with the object code and the nm utility, the back-trace can provide a function call stack.

A buffer of at least max_len unsigned ints.
Size of back-trace buffer.

31.3.8 memdebug_untraced_alloc: Obtain memory from the client OS


#include <oskit/memdebug.h>

void *memdebug_untraced_alloc(oskit_u32_t size, oskit_u32_t align_bits, oskit_u32_t align_ofs);


Obtains memory of the given size and alignment constraints from the client OS. Used by the memdebug library to get the “raw” memory that it tracks.

The size (in bytes) of the chunk to allocate.
The number of low bits of the returned memory chunk address that must match the corresponding bits in align_ofs.
The required offset from natural power-of-two alignment. If align_ofs is zero, then the returned memory block will be naturally aligned on a 2alignbits boundary.

31.3.9 memdebug_untraced_free: Return memory from the client OS


#include <oskit/memdebug.h>

void memdebug_untraced_free(void *ptr, oskit_u32_t size);


Returns the indicated memory with the given size to the client OS. Used by the memdebug library to free the “raw” memory that it tracks.

Memory to free.
The size (in bytes) of the chunk being freed.