This page describes work in progress to add fine-grained bounds checking to GCC's C and C++ front-ends. Interested parties are invited to port to Objective C as well. Please contact Greg McGary, greg@mcgary.org if you wish to assist with development or testing.
Bounded Pointers are easy to understand. GCC augments every pointer datum with two additional pointers that hold the low bound and high bound of the object to which the pointer is seated. Prior to dereference, GCC generates code to test whether the pointer's value lies within the bounds, and if bounds are violated, to generate a machine exception.
Many find the notion of changing the size of a fundamental data type alarming, but for well-formed higher-level C code that uses accurate function prototypes and avoids abusing pointer/integer casts, this is seldom a problem in practice. Even low-level code can use bounded pointers with some extra care.
Basic functionality is present
for Intel x86 using GCC code on the CVS tag
``bounded-pointers-ss-20000730
''. Basic functionality
includes ...
addressof
operator.
(Bounded pointers are also returned by memory allocators such as
malloc
, but that's implemented by the allocator library.)
__bounded
'' and ``__unbounded
,'' and
access to the components of a bounded pointer via new prefix operators
``__ptrvalue
'', ``__ptrlow
'' and
``__ptrhigh
''.
I have tested BPs on a number of packages (see Packages Tested Using Bounded Pointers for
details). I have completed a full (mostly successful) bootstrap of
GCC for LANGUAGES=c
passing
``-fbounded-pointers
'' at all stages (see Bootstrap GCC with Full Bounds Checks for
details).
I have recently committed support for
bounded pointers to the trunk of the GNU C library CVS tree. Intel
x86 is functional. PowerPC is in progress but still incomplete. 90%
of the glibc
testsuite passes in BP mode for Intel x86. C library
support includes thunks for system calls that accept bounded pointer
arguments, check their bounds and pass simple pointers on to the
OS kernel.
My primary focus is on getting GCC changes out of the branch and committed to the CVS trunk.
My secondary focus is on writing documentation which is necessary in order to guide developers and testers.
The most important unfinished bits are:
Bounded pointers enforce data-integrity at the finest possible granularity. Once a pointer is seated to a datum, be it a scalar, array, array element, structure, or structure member, references through that pointer may not exceed the bounds of the datum. Purify won't do this for you. As long as a pointer references valid memory, purify won't protest that your program blew the bounds of an array and started overwriting an adjacent data structure.
Functions having pointers in their return-type/arg-types signature are incompatible between the BP and non-BP modes. In order to prevent unwanted mixing (i.e., calling a function in BP mode when it is defined in non-BP mode, and vice-versa), GCC ``mangles'' the symbols of all BP mode functions that have pointers in their signatures. The presence of BP-mangled symbols causes unwanted mixing to be detected at link time, rather than at runtime where the debug cost is very much higher.
As of this writing, the C function names are mangled by prepending
``__BP_
''. This is subject to change, since using a
suffix might work better with gdb (see Other
Links in the Toolchain). At this time, only function names are so
mangled. It would be better to also mangle the names of global data
structures that contain pointers.
For C++, whose functions are already mangled, I intend to add a boundedness qualifier to the mangling scheme, perhaps adding the letter `X' after the `P' that indicates pointer. C++ does not mangle the type of a function's return value, but in BP mode, this information is essential. The calling convention for returning a bounded-pointer is incompatible with that for returning a single word. A bounded pointer is represented as a three-word struct, so returning one means returning a struct by value, which requires that the caller designate space for the return value and pass a hidden first argument that points to it. The presence of the hidden pointer argument shifts the argument list by one slot, making it incompatible with the non-BP-return case.
Space and time overheads for
bounded-pointer programs are both approx 150%..200% (i.e., 2.5x..3x
slowdown and 2.5x..3x code size increase). A couple years back I
implemented bounded pointers in gcc-2.7.2
with much hackage at
the RTL layer, and using a special BP machine mode (akin to the complex-number
machine modes) that allowed GCC to assign BP components individually
to registers, and to pass/return BPs components in registers. This
version had space and time overhead of only 75%, and that was without
any optimizations to eliminate redundant checks.
This experience leads me to believe that with optimizations to eliminate redundant bounds checks, and with the ability to assign BP components individually to registers, space and time overhead can be brought under 50% (i.e., 1.5x slowdown and 1.5x code size increase).
Bounded pointers do not detect the following errors in memory-usage:
Memory checks are done by Purify or Checker
(Refer to
GCC's -fcheck-memory-usage
option). The checks provided
by bounded pointers and the memory-usage checkers complement each
other nicely without overlap.
Mixing checked and unchecked code is something that's theoretically possible using two mechanisms: (1) explicit qualification of the boundedness of declarations and (2) thunks that translate between bounded-pointer and unbounded-pointer function interfaces.
In practice, the amount of work to properly control mixing is unpredictable. For instance, it's bloody difficult to build bounded-pointer applications of reasonable complexity with an unbounded-pointer C library. On the other hand, it's considerably easier to mix bounded-pointer application code with unbounded-pointer X11 libraries.
I have implemented the beginnings of automatic thunk-generation in GCC, but so far it has only proven useful for building the C torture testsuite in the days before I had a BP-capable C library.
I consider this to be a back-burner project, since I believe that with proper optimization, a 100% bounded-pointer program can be built and run with acceptable space & time overhead. In the absence of a performance justification for mixing unchecked code, the other reason to mix unchecked code is because one has only binaries. As a free-software project, bounded pointers in GCC exist primarily to benefit the free-software community, so I don't intend to go out of my way to accommodate programs that can't be built entirely from source code.
A third reason to mix unchecked code might be to work in stages on converting a large system to become bounded-pointer capable. It would be nice to provide this option, but other things are more important for now, particularly optimizations and broadening the list of supported CPUs.
GCC synthesizes bounds with the
addressof
operation. A data object declared as
``extern
'' with an incomplete type (or with a structure type
containing a flexible array member) has unknown size, but might have
its address taken. Since GCC can't compute the high bound based on an
unknown size, it generates datum foo
's high bound as a
reference to the synthetic symbol ``foo.high_bound
''. If
foo
is defined as initialized data, GCC generates the
label definition of foo.high_bound
immediately following
foo
's initializers. However, if foo
resides
in uninitialized data (BSS or common), GCC cannot do this, and it's
left to the linker to synthesize foo.high_bound
.
I have a small patch to GNU ld that does this for ELF targets.
(Get the ld patch from here)
Bounded pointers introduce two nuisances for debugging:
First, bounded pointers are represented internally as three-member
structures containing simple pointer members for the value, low bound
and high bound. Gdb currently knows nothing about bounded pointers
and treats them according to the information in the symbol table.
Print a pointer variable and you'll see a three member struct.
Attempt to dereference a pointer variable via the expression
``*foo
'', and you'll get an error because gdb thinks foo is
a struct--you must dereference with ``*foo.value
''.
Second, if a function has a pointers as any of its return type or
argument types, its assembler-name is prefixed with
``__BP_
''. Therefore, you need to prefix such function
names when setting breakpoints or printing function addresses.
It would be useful to teach gdb about these two idiosyncrasies of bounded pointers.
You will need a small patch to gdb so that it won't crash starting up on a BP-mode program. (Get the gdb patch from here)
The ``__BP_
'' prefix that is applied to functions
having pointers in their return-type/arg-types signature presents
problems for autoconf. Autoconf tests for the presence of library
functions by creating a tiny test program that compiles and links with
a library. If the test program fails to link, then the function is
considered to be absent from the library and the package supplies a
substitute. The declaration coded into the test program is a phony
one of this form: ``char foo ();
''. If one wishes to
configure with the GCC option ``-fbounded-pointers
'', and
foo
has pointers in its signature, its library definition
will be as ``__BP_foo
'', but the phony declaration will
compile as a reference to the simple ``foo
'' and thus
yield a false negative. A work-around is to always configure with the
non-BP version of a library. I hope that a long-term solution will
come with extensions to autoconf that arrange to get a prototype for
the function under test.
If you wish to help with development and/or testing, you must first
build a baseline. In the examples below, the shell variables
``$..._dir
'' represent the directory names of your
toplevel gcc
, glibc
, ld
and
gdb
trees. The shell variables
``$..._repo
'' hold the names of the GCC and
glibc
CVS repositories. The values of these repository
variables will depend on whether you have write access or have
readonly access through pserver/anoncvs
mode. I'll
assume you know enough about CVS and about configuring and building
GNU packages to adapt the procedure below to fit your environment.
$ mkdir -p $gcc_dir/BUILD $ cd $gcc_dir $ cvs -d $gcc_repo co -rbounded-pointers-ss-20000730 -d src gcc $ cd BUILD $ ../src/configure --prefix=$gcc_dir --enable-languages=c $ make bootstrap $ make install
For convenience, you might wish to install a symlink called
``gcc-bp
'' in one of your bin directories that refers to
$gcc_dir/bin/gcc
.
$ mkdir -p $glibc_dir/BUILD $ cd $glibc_dir $ cvs -d $glibc_repo co -d src libc $ cd BUILD $ env CC=$gcc_dir/bin/gcc ../src/configure --prefix=$glibc_dir \ --enable-bounded --disable-profile --disable-shared $ make $ make install
I recommend ``--disable-profile
'' and
``--disable-shared
'' in order to shorten build time since
you won't need these targets.
I won't give detailed instructions here, because there's nothing out of the ordinary. Download a modern binutils release, or get the code from CVS.
You will need a small patch to GNU ld so that it will synthesize
``foo.high_bound
'' symbols for common & bss symbols. (Get the ld patch from here) The patch
is relative to binutils-2.10
, but will work on
binutils-2.9
as well.
I won't give detailed instructions here, because there's nothing out of the ordinary. Download a modern gdb release, or get the code from CVS.
You will need a small patch to gdb so that it won't crash starting up
on a BP-mode program. Get the gdb
patch from here. The patch is relative to gdb-5.0
,
but will work on gdb-4.18
as well, if you supply the
`-l
' option to patch
to make it more lenient
about whitespace differences.
Now that you have the essentials for working with bounded pointers, here are some suggestions for testing. I present them in order of increasing difficulty. You will be testing three things: (1) correctness of BP-mode code generated by GCC, the correctness of the C library's handling of BPs, and (3) correctness of the code under test. If you wish to focus on debugging the BP implementation in GCC and the GNU C library, then you should test using mature infrastructure packages that have been around for many years. If you test on new code, you're more likely to find bugs in the application, which is of course what the BP feature is designed for, so you are most surely welcome to do that!
This is easy. Just run ``make check
'' after building.
Most tests pass. As for the rest, pick one and debug it.
Here's how to run the GCC C torture tests in BP mode:
$ make check-gcc RUNTESTFLAGS="--tool_opts=\"-g -fbounded-pointers -static \ -B$glibc_build_dir/csu/ -L$glibc_build_dir\""
Remember that ``$..._dir
'' variables represent directory
names from your system. Note the use of
``-B$glibc_build_dir/csu/
'' to get bcrt1.o
,
and ``-L$glibc_build_dir
'' to get libraries. Both of
these options refer to your C library build directories, not to the
directories in which you installed the C library. This is
intentional. The only thing you really need from the install tree is
the header tree in ``$glibc_dir/include
''. For the rest,
it is more convenient to get the files directly from the build tree,
so that when you rebuild glibc
after fixing a bug, you
can avoid the install step. Naturally, if you change a public header
file, you'll need to do the install, but this happens much less
frequently.
Pick a favorite package and have at it. Don't forget to build a BP version of any extra libraries the package requires.
Because of the problems with autoconf
mentioned above, the best workaround is to configure with the static
non-BP version of the C library you built alongside the BP version.
Your installed C library will invariably be an older version of
glibc
, and will yield different configuration results, so
you don't want to use it.
I use a couple of ``wrapper'' script to prefix the
configure
command that gives me a suitable environment
for using the newly-built C library.
This one is called ``ubpenv
'':
#!/bin/bash export CC=$gcc_dir/bin/gcc export LDFLAGS="-static -B$glibc_build_dir/csu/ -L$glibc_build_dir" export CFLAGS="-isystem $glibc_dir/include -O2" "$@"
This one is called ``bpenv
'', and differs only in the
value of CFLAGS
:
#!/bin/bash export CC=$gcc_dir/bin/gcc export LDFLAGS="-static -B$glibc_build_dir/csu/ -L$glibc_build_dir" export CFLAGS="-isystem $glibc_dir/include -fbounded-pointers \ -fno-optimize-sibling-calls -O2 -g" "$@"
I recommend that you turn off sibling-call optimizations in order to preserve complete call traces and avoid surprises while debugging.
In order to override the configured value of CFLAGS, you need to build like so:
$ bpenv eval make 'CFLAGS="$CFLAGS"'
To save some typing, I have a third script called ``bpmake
'':
#!/bin/bash bpenv eval make 'CC="$CC"' 'CFLAGS="$CFLAGS"' 'LDFLAGS="$LDFLAGS"' "$@"
With these scripts, the sequence for building and testing a GNU package in BP mode is this:
$ ubpenv ./configure $ bpmake $ bpmake check
The bootstrap procedure outlined below depends on already having a
BP-capable compiler installed, and is performed on the GCC source tree
at CVS tag ``bounded-pointers-ss-20000730
''. This procedure
doesn't produce a GCC that's particularly useful, since it's so much
slower. This is purely a testing exercise in order to expose bounds
violations in GCC, and to validate the correctness of bounds-checked
code.
The host compiler is gcc-bp
, an ordinary unchecked
program that produces a checked stage1. The stage1 compiler is fully
bounds checked, and so runs like a pig on quaaludes while producing
the stage2 compiler. The stage2 compiler is a companion pig on
quaaludes that produces a third drugged pig. We do the final binary
compare on the second- and third-stage pigs, and use the third-stage
pig to run the test suite.
There are some potholes along the road that you'll need to steer around:
makeinfo
, install-info
, and
texindex
don't link for lack of a BP version of
libz.a
. We don't need texinfo
, so
we can just ignore it.
gettext
implementation in
gcc/intl/libintl.a
conflicts with
glibc
's, so we must configure to ignore GCC's.
First, you must supplement the command-line in the
bpmake
script with these extra arguments:
'BOOT_CFLAGS="$CFLAGS"' 'BOOT_LDFLAGS="$LDFLAGS"' 'SYSTEM_HEADER_DIR="$glibc_dir/include"'
With that done, this procedure does the trick:
$ ubpenv ./configure --without-included-gettext --enable-languages=c $ bpmake all-libiberty $ bpmake -C gcc $ bpmake -C gcc stage1 bootstrap2
The second and third stages compare cleanly. Unfortunately,
running the test suite yields these extra failures that did not
appear for the installed gcc-bp
:
FAIL: gcc.c-torture/compile/981001-2.c, -O0 FAIL: gcc.c-torture/compile/981001-2.c, -O1 FAIL: gcc.c-torture/compile/981001-2.c, -O2 FAIL: gcc.c-torture/compile/981001-2.c, -O3 -fomit-frame-pointer FAIL: gcc.c-torture/compile/981001-2.c, -O3 -g FAIL: gcc.c-torture/compile/981001-2.c, -O3 -fssa FAIL: gcc.c-torture/compile/981001-2.c, -Os FAIL: gcc.c-torture/execute/990117-1.c execution, -O3 -fomit-frame-pointer FAIL: gcc.c-torture/execute/990117-1.c execution, -O3 -g FAIL: gcc.c-torture/execute/990117-1.c execution, -O3 -fssa FAIL: gcc.c-torture/execute/ieee/minuszero.c execution, -O1 FAIL: gcc.c-torture/execute/ieee/minuszero.c execution, -O2 FAIL: gcc.c-torture/execute/ieee/minuszero.c execution, -O3 -fomit-frame-pointer FAIL: gcc.c-torture/execute/ieee/minuszero.c execution, -O3 -fomit-frame-pointer -funroll-loops FAIL: gcc.c-torture/execute/ieee/minuszero.c execution, -O3 -fomit-frame-pointer -funroll-all-loops -finline-functions FAIL: gcc.c-torture/execute/ieee/minuszero.c execution, -O3 -g FAIL: gcc.c-torture/execute/ieee/minuszero.c execution, -O3 -fssa FAIL: gcc.c-torture/execute/ieee/minuszero.c execution, -Os FAIL: gcc.dg/20000419-2.c (test for excess errors) FAIL: alias-1.c FAIL: wkali-1.c FAIL: wkali-2a.o FAIL: gcc.misc-tests/gcov-1.c (test for excess errors) WARNING: gcc.misc-tests/gcov-1.c compilation failed to produce executable FAIL: gcov-1.c:1:is 4:should be 11 FAIL: gcov-1.c:1:is 5:should be 10 FAIL: gcov-1.c:1:is 7:should be 1 FAIL: gcc.misc-tests/gcov-2.c (test for excess errors) (PRMS 8294) WARNING: gcc.misc-tests/gcov-2.c compilation failed to produce executable
Even so, it's not so bad for an intoxicated pig.
Below is a list of results for packages tested with bounds checking. Unless otherwise noted, tests were done by me (Greg).
Bounds violations exposed:
regex.c
: This patch
was required.
100% of the test suite passes after fixing the bugs listed above. (However, the maintainer admits that the test suite is hardly comprehensive.) Fixes appear in 3.0.6.
Bounds violations exposed:
regex.c
: This patch
was required.
pr
: init_header
wrote past the end
of a string buffer for pages with column-width less than 22 characters.
pr
: store_columns
read a column descriptor
that was one past the end of the array of columns.
tail
: pipe_lines
read past the
beginning of a string buffer when given an empty input file.
100% of the test suite passes after fixing the bugs listed above. Fixes appear in 2.0h
Bounds violations exposed:
regex.c
: This patch
was required.
100% of the test suite passes after fixing the bugs listed above.
No bounds violations exposed. 100% of the test suite passes.
No bounds violations exposed. 100% of the test suite passes.
Bounds violations exposed:
fixwrites
: main
read past the
beginning of a string buffer when presented with an empty line.
100% of the test suite passes after fixing the bug listed above
and compiling with ``gcc ... -fno-strict-aliasing
''.
A strict-aliasing bug for i586 and i686 caused two assertion failures
in kpathsea
.
Bounds violations exposed:
bfd/archive.c
: Many calls to sprintf
for filling fields of struct ar_hdr
write
a NUL-terminator one beyond the end of the field.
ld/ldlang.c
: Missing prototype for
walk_wild.c
caused callback pointer to be erroneously
treated as bounded. GCC should be fixed to handle this, since
the non-prototype definition of walk_wild
appeared
before its use.
100% of the test suite passes after fixing the bug listed above.
bounded-pointers-ss-20000730
Preliminary results are described at Bootstrap
GCC with Full Bounds Checks. Later, I'll turn the bounds-checked
gcc loose on a recent release, such as gcc-2.95.2
and
see if any bounds violations occur.
Bounds violations exposed:
regex.c
: This patch
was required.
Testing and fixing is in progress...
Bounds violations exposed:
regex.c
: This patch
was required.
mkid
: assert_hits
read past the
beginning of an array.
Testing and fixing is in progress...
Here is a list of bugs known to exist for bounded pointer mode in GCC and in the GNU C library, as well as some commonly found problems in applications:
GCC generates bad bounds-checking code causing spurious bounds
violations in nss_parse_service_list
, which is used
internally by the GNU C library's name-service switch. The cause is
unknown.
GCC generates bad bounds-checking code causing spurious bounds
violations in canonicalize
, which is used internally by
the GNU C library's character-set conversion code. The cause is
unknown.
Programs that use their own version of GNU regex.c
are missing some special handling for BPs in
EXTEND_BUFFER
. Get the
patch for regex.c
from here.
Threaded applications with linuxthreads
are
unusable in BP mode. So far, gdb
has been useless for
debugging these, so this will take some time and head-scratching to
resolve.
Most of the bounded pointers implementation is machine independent, both in GCC and in the C library. These are the machine-dependent parts:
Bounded pointers depend on conditional trap patterns being defined
in the machine description. Some machine descriptions already have
them, namely SPARC, rs6000 (PowerPC), m68k, m88k and i960. All of
these have machine instructions that implement conditional traps with
one instruction. Beginning with ISA-II, MIPS has conditional trap
instructions as well, but its GCC machine description so far lacks
them. Intel x86 has no conditional trap instructions, but I defined
conditional trap patterns that expand to primitive instructions to
test and conditionally jump around an ``int 5
''
instruction. If the CPU you wish to support has no conditional trap
instructions, you should define pseudo conditional traps as I have
done for x86.
Conditional traps are important for the sake of optimization. Without them, GCC would need to emit conditional branches as RTL, whose presence would artificially partition basic blocks and inhibit other optimizations. Also conditional trap RTL expressions are readily identifiable and thus more conveniently checked for redundancies that can be eliminated.
Most CPUs on which the GNU C library runs define some functions in
assembler which have pointers in their signatures. Some are coded in
assembler because they are performance critical, such as the memory
and string functions (bcopy
, bzero
,
memcpy
, memcmp
, memset
,
strchr
, strcpy
, strcmp
,
strlen
, strtok
, etc), primitives for
multi-precision arithmetic (add_n
, addmul_1
,
mul_1
, sub_n
, submul_1
,
lshift
, rshift
), and primitives for
floating-point math (frexp
, frexpf
,
freexpl
, remquo
, remquof
,
remquol
, sincos
, sincosf
,
sincosl
). Some are coded in assembler because they have
special semantics that can't be achieved with plain C, namely
setjmp
and longjmp
. Finally, some have
special interfaces to the kernel or C runtime, namely
brk
, clone
and the startup code.
The assembler functions need to conditionally compile in BP and non-BP modes. In BP mode, they must accommodate the calling convention where pointer arguments and return value are structs passed by value, and they must check the bounds of their arguments. The best way to proceed is to study what's already been done for Intel x86, a CISC target, and for PowerPC, a RISC target.
Again, the best way to proceed is to study what's already been done for Intel x86 and (soon) for PowerPC.
Sorry, nothing yet... This stuff properly belongs in either the GCC manual or the GCC ``Internal Representation'' document.