4 ms·

Great article, thanks - one question I couldn't see answered there is, what do you do when you want to set more than four breakpoints at once?

saagarjha 3y ago
Use software breakpoints (which are mentioned but not described, the short story for those is you overwrite the address you care about with an illegal instruction and execution traps when it encounters that code, and then you undo it to continue).
ithkuil 3y ago
You use software breakpoints. Basically you overwrite the instruction you want to break at with a breakpoint instruction (e.g. int 3 on x86). This will cause the process to trap and the OS will then let the debugger process know about out somehow, e.g. via the SIGTRAP signal on Unix. The debugger then replaces the int 3 opcode (which is a single byte conveniently) with the first byte of the original instruction so that the execution can continue.
- hinoki 3y ago
  If you revert the int 3 to the original instruction’s byte, when do you put it back? The breakpoint could still be active. In a trivial example, the breaking instruction could be a jump to itself, which you’d expect to immediately break into the debugger again. I thought the debugger had to emulate the instruction instead, but it’s not like I’ve ever implemented one…
  - ithkuil 3y ago
    Emulation is an option, rotating hardware debug registers is another option, detecting self-jumps is another option. I really only implemented a debugger for the esp8266 and it was just good enough for me and my team to get our job done so it didn't handle many edge cases like that
  - i_don_t_know 3y ago
    I believe when you resume the debugger, you can tell the process/thread to single-step over one instruction. So it's something like this: 1. Overwrite instruction with int 3. 2. When you hit the breakpoint, restore the original instruction. 3. Single-step over the original instruction by changing the thread's EFlags (Intel). 4. Restore the breakpoint with int 3. 5. Resume normally.
    - hinoki 3y ago
      Wouldn’t that race against any other thread in the process? I guess you could stop all threads when you hit the breakpoint and start them again after you restore the breakpoint, but the synchronisation of that would be really tricky too.
      - Veserv 3y ago
        Yes. And yes that is one of the ways to solve it. You could also do something like have a clean mapping table (i.e. the code with no breakpoints installed) that you install for just the thread doing the step. You then revert back to the normal mapping table with the breakpoint after the step. As you are only modifying the executable section, as long as you are not using self-modifying code, there should be no data inconsistency with having a multiple copys of the executable transiently.
  - timmisiak 3y ago
    Yes, software breakpoints are difficult to get correct (the main reason why I started with hardware breakpoints). It gets more complicated with kernel debugging, where a single step (trap flag) could get pre-empted by an interrupt handler. And you can't always single-step a CPU and leave all other CPUs frozen.
  - spc476 3y ago
    As i_don_t_know stated, if the CPU has the ability to single step an instruction, you use that. Otherwise: * Restore the original instruction byte. * Find the next instruction, and set a temporary software breakpoint there. * Resume the one instruction * Restore the original instruction byte at the temporary software breakpoint. * Set the software breakpoint in the original instruction * Resume running The other thing to keep in mind is dealing with JMP, CALL and conditional branch instructions. It can get pretty messy pretty quick, which is why I find low level debuggers on old 8-bit CPUs a marvel as they had to deal with only software breakpoints.
vinge 3y ago
Others have already mentioned software breakpoints where the instruction is replaced, another option is to run the code in an emulator that supports a virtually unlimited set of breakpoints. For example, using QEMU with its GDB stub.

saagarjha 3y ago

Use software breakpoints (which are mentioned but not described, the short story for those is you overwrite the address you care about with an illegal instruction and execution traps when it encounters that code, and then you undo it to continue).

ithkuil 3y ago

You use software breakpoints. Basically you overwrite the instruction you want to break at with a breakpoint instruction (e.g. int 3 on x86). This will cause the process to trap and the OS will then let the debugger process know about out somehow, e.g. via the SIGTRAP signal on Unix. The debugger then replaces the int 3 opcode (which is a single byte conveniently) with the first byte of the original instruction so that the execution can continue.

hinoki 3y ago
If you revert the int 3 to the original instruction’s byte, when do you put it back? The breakpoint could still be active. In a trivial example, the breaking instruction could be a jump to itself, which you’d expect to immediately break into the debugger again. I thought the debugger had to emulate the instruction instead, but it’s not like I’ve ever implemented one…
- ithkuil 3y ago
  Emulation is an option, rotating hardware debug registers is another option, detecting self-jumps is another option. I really only implemented a debugger for the esp8266 and it was just good enough for me and my team to get our job done so it didn't handle many edge cases like that
- i_don_t_know 3y ago
  I believe when you resume the debugger, you can tell the process/thread to single-step over one instruction. So it's something like this: 1. Overwrite instruction with int 3. 2. When you hit the breakpoint, restore the original instruction. 3. Single-step over the original instruction by changing the thread's EFlags (Intel). 4. Restore the breakpoint with int 3. 5. Resume normally.
  - hinoki 3y ago
    Wouldn’t that race against any other thread in the process? I guess you could stop all threads when you hit the breakpoint and start them again after you restore the breakpoint, but the synchronisation of that would be really tricky too.
    - Veserv 3y ago
      Yes. And yes that is one of the ways to solve it. You could also do something like have a clean mapping table (i.e. the code with no breakpoints installed) that you install for just the thread doing the step. You then revert back to the normal mapping table with the breakpoint after the step. As you are only modifying the executable section, as long as you are not using self-modifying code, there should be no data inconsistency with having a multiple copys of the executable transiently.
- timmisiak 3y ago
  Yes, software breakpoints are difficult to get correct (the main reason why I started with hardware breakpoints). It gets more complicated with kernel debugging, where a single step (trap flag) could get pre-empted by an interrupt handler. And you can't always single-step a CPU and leave all other CPUs frozen.
- spc476 3y ago
  As i_don_t_know stated, if the CPU has the ability to single step an instruction, you use that. Otherwise: * Restore the original instruction byte. * Find the next instruction, and set a temporary software breakpoint there. * Resume the one instruction * Restore the original instruction byte at the temporary software breakpoint. * Set the software breakpoint in the original instruction * Resume running The other thing to keep in mind is dealing with JMP, CALL and conditional branch instructions. It can get pretty messy pretty quick, which is why I find low level debuggers on old 8-bit CPUs a marvel as they had to deal with only software breakpoints.

vinge 3y ago

Others have already mentioned software breakpoints where the instruction is replaced, another option is to run the code in an emulator that supports a virtually unlimited set of breakpoints. For example, using QEMU with its GDB stub.