GDB's file `remote.c' talks a serial protocol to code that runs in the target system. GDB provides several sample "stubs" that can be integrated into target programs or operating systems for this purpose; they are named `*-stub.c'.
The GDB user's manual describes how to put such a stub into your target code. What follows is a discussion of integrating the SPARC stub into a complicated operating system (rather than a simple program), by Stu Grossman, the author of this stub.
The trap handling code in the stub assumes the following upon entry to trap_low:
As long as your trap handler can guarantee those conditions, then there is no
reason why you shouldn't be able to `share' traps with the stub. The stub has
no requirement that it be jumped to directly from the hardware trap vector.
That is why it calls exceptionHandler(), which is provided by the external
environment. For instance, this could setup the hardware traps to actually
execute code which calls the stub first, and then transfers to its own trap
handler.
For the most point, there probably won't be much of an issue with `sharing'
traps, as the traps we use are usually not used by the kernel, and often
indicate unrecoverable error conditions. Anyway, this is all controlled by a
table, and is trivial to modify.
The most important trap for us is for ta 1. Without that, we
can't single step or do breakpoints. Everything else is unnecessary
for the proper operation of the debugger/stub.
From reading the stub, it's probably not obvious how breakpoints work. They are simply done by deposit/examine operations from GDB.