On-Chip Debugging

FPGA bugs are often invisible at the pins. A state machine may enter an unexpected state, a FIFO may underflow for one clock, or a packet parser may reject a frame because one internal bit arrived late. On-chip debugging adds observability inside the FPGA so you can capture those internal signals on real hardware.

Learning Objectives

By the end of this lesson, you should be able to:

choose simulation, external probing, or on-chip capture for the right failure;
select useful debug signals without exhausting memory;
define meaningful triggers and capture windows;
avoid debug logic that hides timing-sensitive bugs;
plan design-for-debug features before board bring-up.

Debugging Order

Start with the cheapest evidence. Simulation is usually faster than repeated synthesis, bitstream generation, and hardware testing.

flowchart TD FAIL["Failing behavior"] --> SIMQ["Can simulation reproduce it?"] SIMQ -->|Yes| SIM["Fix with testbench and waveform"] SIMQ -->|No| BOARD["Check board inputs, clocks, reset, pins"] BOARD --> OBS["Add observability"] OBS --> CAP["ILA, SignalTap, UART trace, LEDs"] CAP --> TRIG["Trigger on meaningful event"] TRIG --> CMP["Compare capture with expected sequence"]

Use hardware debug for board-specific timing, real I/O behavior, clocking, reset sequencing, integration faults, or bugs that the testbench does not yet model.

Debug Tools

Tool	Best use	Limitation
LED or GPIO pin	heartbeat, state bit, simple trigger	very low bandwidth
UART trace	counters, status snapshots, slow logs	changes timing and consumes logic
Integrated logic analyzer	internal waveforms around a trigger	limited probes and sample depth
External logic analyzer	physical pins, protocols, analog-adjacent timing	cannot see internal nets
Simulation assertions	expected protocol and safety rules	only as good as stimulus and models
Formal checks	exhaustive control-path properties	needs precise properties and bounded scope

The right tool depends on the question. If you need to know whether rx_valid asserted before fifo_full, an integrated logic analyzer is useful. If you need to know whether a pin violates an electrical level, use lab equipment.

Design for Debug

Debug is easier when the RTL already exposes useful structure:

named FSM states rather than unreadable encoded constants;
sticky error flags for rare failures;
counters for dropped packets, FIFO overflow, timeout, and checksum errors;
heartbeat counters per clock domain;
valid/ready signals at every streaming boundary;
optional debug muxes with synthesis guards;
spare pins or headers reserved for bring-up;
build option to include or remove debug cores.

Use a synthesis parameter or build define so debug logic can be included intentionally:

generate
if (ENABLE_DEBUG) begin : gen_debug
    assign debug_bus = {state, fifo_full, fifo_empty, rx_valid, tx_ready};
end
endgenerate

Trigger and Capture Strategy

Do not capture randomly. A good trigger describes the exact event that separates normal behavior from failure.

Symptom	Useful trigger	Signals to capture
FIFO data repeats	`rd_en && empty`	read/write pointers, empty, full, data, valid
Packet disappears	checksum fail pulse	parser state, byte count, CRC, valid/ready
Bus read hangs	bus valid without ready timeout	address, write/read, ready, FSM state
Motor PWM glitches	period wrap and shadow load	counter, active duty, shadow duty, output
CDC failure suspected	synchronizer edge mismatch	source toggle, synchronized toggle, event pulse

Capture some samples before and after the trigger. The pre-trigger window shows how the design reached the failure.

Worked Example: FIFO Underflow

Symptom: an output sample occasionally repeats. A plausible failure is that the consumer reads while the FIFO is empty.

Useful trigger:

wire fifo_underflow = rd_en && empty;

Signals to capture:

rd_en, empty, rd_data;
read pointer and write pointer;
synchronized write pointer in the read clock domain;
consumer valid and ready;
reset and clock-domain heartbeat.

Expected behavior: rd_en must never be high when empty is high. If the capture shows rd_en && empty, the consumer handshake is wrong or the empty flag is late. If it never happens, the repeated data may come from pointer synchronization, output register enable, or downstream backpressure.

Interpreting Captures

Treat captured waveforms as evidence, not proof by themselves. Check:

are all captured signals in the same clock domain;
is the ILA clock the same as the probed logic clock;
did probe insertion change timing;
is the trigger condition too broad or too late;
did synthesis optimize away a signal before probing;
does the capture match the RTL version currently loaded on the board.

If adding probes makes the bug disappear, compare timing reports before and after probe insertion. The debug core may have changed placement, fanout, or routing delay.

Practical Bring-Up Sequence

Confirm power rails, configuration, clock, and reset.
Load a known-good LED heartbeat bitstream.
Verify pin constraints with simple I/O patterns.
Run simulation for the failing feature.
Add a small set of targeted probes.
Trigger on the failure event, not on time since reset.
Save the capture, bitstream hash, commit, and constraints version.
Fix the RTL and reproduce the fix in simulation before closing the issue.

Common Mistakes

Debugging hardware before writing a focused testbench.
Capturing too many signals and too few samples.
Probing signals from multiple clock domains without marking their domains.
Forgetting that debug cores consume BRAM, routing, and timing margin.
Leaving debug-only behavior enabled in production builds.
Trusting an old bitstream after the RTL has changed.

Summary

On-chip debugging is most effective when it is targeted. Reproduce failures in simulation when possible, add observability for the specific event that matters, capture enough context around the trigger, and account for the resources and timing changes caused by debug logic.

Next: Introduction to Verilog HDL.

Mind Map

mindmap root((On Chip Debug)) Core concept Observe internal nets Trigger captures event Simulation first Probes cost resources Applications Board bring up Rare integration bugs FIFO faults Protocol failures Timing-sensitive issues Calculations Capture time equals samples over clock BRAM use grows with probes times depth Trigger before failure Design rules Probe same clock domain Use sticky flags Keep debug optional Save bitstream version Use focused triggers Practical checks Clock and reset alive Constraints match pins Compare with expected waveform Check probe timing impact Reproduce fix in sim Common mistakes No testbench Too many probes Wrong ILA clock Debug hides bug Old bitstream tested