Debugging Tools for Embedded Systems

Why This Matters

Debugging is where embedded systems design moves from theory to reality, and where most of your development time will actually be spent. You're being tested not just on knowing what tools exist, but on understanding when and why to reach for each one. The core concepts here revolve around observability (how do you see what's happening inside a system?), controllability (how do you pause, step, and manipulate execution?), and the fundamental distinction between hardware-level and software-level debugging approaches.

These tools represent different trade-offs between intrusiveness, cost, timing accuracy, and ease of use. A logic analyzer gives you non-intrusive timing data but no code context; an IDE debugger gives you source-level visibility but may alter real-time behavior. Don't just memorize tool names. Know what category of problem each tool solves and what compromises it makes.

---

Hardware Interface Debuggers

These tools connect directly to the target processor through dedicated debug interfaces, giving you deep access to internal state without relying on the running software.

The key principle: dedicated hardware debug ports bypass normal execution, allowing inspection even when software is crashed or unresponsive.

In-Circuit Emulators (ICE)

• Replaces or intercepts the target CPU, providing complete control over execution while the rest of the system "thinks" it's running normally
• Real-time emulation means timing-critical code can be debugged without the probe effect that slows software debuggers
• Architecture-specific hardware makes ICE expensive, but it's invaluable for debugging interrupt handlers, bootloaders, and bare-metal code where no OS or runtime support exists

ICE tools have become less common as on-chip debug modules have improved, but they still appear in legacy systems and in situations where you need cycle-accurate visibility that on-chip debug can't provide.

JTAG Debuggers

JTAG (IEEE 1149.1) is the industry-standard debug interface found on nearly every modern microcontroller and processor. It works through an on-chip debug module rather than replacing the CPU.

• Boundary-scan capability allows testing solder joints and PCB connections, bridging hardware and software debugging. This was actually JTAG's original purpose before it became the dominant software debug interface.
• Direct register and memory access enables inspection of peripheral configurations, stack contents, and CPU state at any breakpoint
• Serial Wire Debug (SWD) is a two-pin alternative to JTAG common on ARM Cortex-M devices. It offers similar debug capabilities with fewer pins, which matters on small packages.

---

Signal Analysis Tools

When bugs manifest as timing glitches, protocol errors, or electrical noise, you need tools that observe actual signals rather than software state.

These tools are non-intrusive: they watch without affecting execution timing, making them essential for real-time system debugging.

Logic Analyzers

• Captures digital signals across many channels simultaneously, which is essential for debugging buses like SPI, I2C, UART, and parallel interfaces
• Protocol decoding translates raw bit patterns into meaningful transactions, showing you exactly where communication breaks down (e.g., a missing ACK on I2C, or a chip-select timing violation on SPI)
• Complex triggering lets you trigger on specific data patterns or signal sequences to catch intermittent bugs that simple observation misses

Logic analyzers only see signals as high or low. They tell you what data was transmitted and when, but they can't tell you anything about signal quality.

Oscilloscopes

• Visualizes analog signal characteristics like voltage levels, rise/fall times, ringing, and noise that logic analyzers completely ignore
• Signal integrity analysis reveals hardware problems such as impedance mismatches, ground bounce, and power supply droop
• Mixed-signal oscilloscopes (MSOs) combine analog and digital channels on the same timebase, letting you correlate software events with electrical behavior

---

Software-Based Debugging

These approaches use the processor's own execution environment to provide debug information, trading some real-time accuracy for convenience and low cost.

Software debugging tools are intrusive: they consume CPU cycles, memory, and I/O resources, which can mask or create timing-sensitive bugs. This is called the probe effect.

Serial/UART Debugging

Printf-style debugging over a serial port is the simplest and most universal embedded debug technique. It requires minimal hardware (often just two GPIO pins and a USB-to-serial adapter) and works on virtually any project.

The trade-off is timing impact. String formatting and transmission take real time. At 115200 baud, transmitting a 50-character debug message takes roughly 4.3 ms. That delay can hide race conditions or cause new ones. You need to be conscious of this whenever you're debugging time-sensitive code.

Semihosting is a related technique where debug output is redirected through the JTAG/SWD connection to the host PC. It avoids needing a UART pin but is even slower because each output call halts the processor briefly.

Integrated Development Environment (IDE) Debuggers

• Source-level debugging maps machine state back to your C/C++ code with variable inspection, call stack visualization, and peripheral register views
• Breakpoints and step execution let you pause at specific lines and walk through logic, but they halt the entire system, which disrupts any real-time behavior
• Integration with build tools streamlines the edit-compile-debug cycle

One thing to watch for: compiler optimizations can make IDE debugging confusing. Variables may be optimized out, code may be reordered, and stepping through optimized code won't follow your source line by line. Many developers debug with optimizations disabled ($-O0$), but this changes timing behavior, which brings us back to the probe effect.

---

Execution Control Mechanisms

These are specific capabilities, often provided through JTAG or IDE debuggers, that let you precisely control and observe program execution.

The distinction between breakpoints and watchpoints is fundamental: breakpoints trigger on code location, watchpoints trigger on data access.

Hardware Breakpoints

• Pauses execution when the program counter reaches a specific address, implemented in dedicated comparator hardware inside the debug unit
• Limited quantity (typically 2-8 per core) because each one requires a physical comparator in silicon
• Works on any memory type, including read-only flash, unlike software breakpoints that must overwrite an instruction with a trap/break instruction

Software breakpoints, by contrast, are unlimited in number because they just replace an instruction in RAM with a special trap opcode (like ARM's BKPT). But they can't be used in flash memory without a flash write cycle, and they can't be placed in code that checksums itself.

Watchpoints

• Triggers when a specific memory location is read or written, making them essential for catching memory corruption
• Data breakpoints help identify who is modifying a variable when the bug isn't where the variable is declared but somewhere else entirely
• Tracks down pointer errors and buffer overflows by watching memory regions that shouldn't be accessed

Watchpoints are also implemented in hardware (using the same debug unit as hardware breakpoints), so they share the same limited count. On an ARM Cortex-M4, for example, you typically get 4 watchpoint comparators.

---

Execution Analysis Tools

These tools record and analyze program behavior over time, providing insight into performance, memory usage, and execution history.

Post-mortem and statistical analysis complements real-time debugging. Some bugs only appear under load or after extended operation.

Trace Tools

Trace captures a continuous record of instruction execution, showing you the path the program took rather than just where it stopped.

• Essential for debugging crashes where the failure point doesn't reveal the root cause that happened hundreds or thousands of instructions earlier
• Performance profiling identifies hot spots and bottlenecks by showing where the CPU spends its cycles
• ARM's ETM (Embedded Trace Macrocell) and ITM (Instrumentation Trace Macrocell) are common implementations. ETM provides full instruction trace; ITM provides lightweight software-instrumented trace with much lower bandwidth requirements.

Trace requires a high-bandwidth debug probe (like a J-Trace or ULINK Pro) and a trace port on the target board. Not every board breaks out the trace pins, so check your hardware before counting on this capability.

Memory Analyzers

• Tracks heap allocations and deallocations to detect leaks where memory is allocated but never freed
• Stack usage analysis determines worst-case stack depth, which is critical for sizing stack space in resource-constrained systems. A common technique is "stack painting" (filling the stack with a known pattern at startup, then checking how much was overwritten).
• Buffer overflow detection instruments memory accesses to catch writes beyond allocated boundaries

On resource-constrained targets, full memory analysis tools (like Valgrind on Linux) may not be available. In those cases, you rely on lighter-weight approaches: stack painting, heap wrappers that track allocations, or MPU (Memory Protection Unit) configuration to trap illegal accesses.

---

Quick Reference Table

---

Self-Check Questions

1. You have a bug where a global variable is being corrupted, but you don't know which function is writing to it. Which debugging mechanism would you use, and why is it more appropriate than a breakpoint?

2. Compare JTAG debuggers and serial/UART debugging: what can JTAG do that serial cannot, and in what situations might you prefer serial debugging despite its limitations?

3. A system works perfectly when you step through it in the IDE debugger but fails when running at full speed. Which category of tools would help diagnose this, and what does this symptom suggest about the bug?

4. Your embedded system crashes after running for several hours. The crash location varies each time. Which two tools from different categories would you combine to investigate, and what would each reveal?

5. Explain why hardware breakpoints are limited in number while software breakpoints are not. In what situation would you need a hardware breakpoint rather than a software one?