Hardware breakpoints are quite useful while developing user or kernel space software. An interesting feature, which makes them be also known as memory breakpoints, is interrupting execution whenever the processor executes, reads or writes a specific virtual address. This feature is not available in software breakpoints.
Once a breakpoint is hit, there might be cases in which we want the execution to stop, so single-stepping or memory analysis is possible; and cases in which we want traces to be generated without a full-stop. I won’t make a distinction between these use cases, given that the underlying mechanism is the same.
As the name suggests, help from the processor is needed for them to work; and implementation depends on each architecture. Contrary to software breakpoints, hardware breakpoints are a scarce resource. In the case of x86 / x86_64, only a few processor registers called from DR0 to DR7 are available to mange breakpoint addresses, configuration and status. The use of these registers requires privileged mode.
It is not necessary to manage debug registers manually: the Linux kernel provides an API for that (see include/linux/linux/hw_breakpoint.h). Kgdb (kernel/debug) is probably its most known user. The good news is that our own kernel modules can use it too. We will see how.
I’ve developed a simple toy application that loads a kernel module and exposes a character device. A special SAMODULE_IOCTL_TEST ioctl can be performed on the character device and is handled as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
static long unlocked_ioctl(struct file* f, unsigned int cmd, unsigned long arg) { ... struct perf_event_attr attr; struct perf_event** bps; ... switch(cmd) { case SAMODULE_IOCTL_TEST: SM_PRINTF("SAMODULE_IOCTL_TEST\n"); hw_breakpoint_init(&attr); attr.bp_addr = (unsigned long)asm_test_function(); attr.bp_len = HW_BREAKPOINT_LEN_8; attr.bp_type = HW_BREAKPOINT_X; bps = register_wide_hw_breakpoint(&attr, bp_handler, NULL); SM_PRINTF("asm_test_function() = %ld\n", asm_test_function()); unregister_wide_hw_breakpoint(bps); break; } ... } |
When the ioctl is handled, a hardware breakpoint is set. The breakpoint will get triggered upon the execution of a specific memory address (returned by asm_test_function) and the bp_handler callback will be invoked immediately after.
Here it’s the callback code:
|
1 2 3 4 5 |
static void bp_handler(struct perf_event* pe, struct perf_sample_data* psd, struct pt_regs* regs) { SM_PRINTF("bp_handler - begin"); SM_PRINTF("bp_handler - IP: 0x%lx\n", regs->ip); SM_PRINTF("bp_handler - end"); } |
You can see there how registers information is available to the callback. In this case, the RIP value is printed.
perf_events are the key component of the hardware breakpoints API in the Linux kernel. Through a perf_event_attr structure, it’s possible to set the breakpoint address, type (execute, read or write) and bytes length filter.
At the implementation level, these breakpoints are set on all CPUs at once. When a CPU hits a breakpoint, the debug entry is looked up in the Interrupt Dispatch Table (IDT). The kernel handles this interruption through the do_debug routine (arch/x86/kernel/traps.c) and a perf machinery is then involved to dispatch registered callbacks.