MIPI CSI-2 from an FPGA on Jetson Orin — nvcsi deskew and custom source bring-up

Using an FPGA as a MIPI CSI-2 source for Jetson Orin is a legitimate approach for custom sensor interfaces, synthetic test patterns, FPGA-based vision preprocessing, and multi-sensor aggregation. The bring-up is harder than with off-the-shelf camera sensors because FPGA MIPI IP implementations vary widely in protocol correctness — especially around deskew calibration and timing margins that silicon sensors handle automatically.

Key Insights

• Deskew calibration is mandatory for lanes above 1.5Gbps — Jetson NVCSI requires it; FPGA MIPI IP that skips it will fail at higher data rates even if the signal looks clean on an oscilloscope
• HS settle time is the most common FPGA timing error — too short and NVCSI fails to enter High Speed mode; too long wastes bandwidth but works
• Short frame errors mean line count or line length doesn’t match DT — the FPGA must output exactly the number of pixels and lines specified in the device tree sensor mode
• Use virtual channels to aggregate multiple FPGA video streams — NVCSI supports VC0–VC3; a single FPGA can output 4 independent streams on one physical MIPI link
• NVCSI deskew can be disabled in the DT for bring-up debugging — helps isolate whether failures are calibration-related or data-format-related

MIPI CSI-2 protocol requirements for FPGA sources

Your FPGA MIPI TX must implement:

1. LP-to-HS transition (Lane Start sequence)
   ├─ LP-11 (both lines high) for ≥100ns
   ├─ LP-01 (D- high, D+ low)
   ├─ LP-00 (both lines low)
   └─ HS preamble sequence (0x00 + 0xB8 sync word)

2. Deskew calibration (required for rates >1.5Gbps)
   └─ Transmit deskew code: 0xBC pattern per lane

3. Packet structure (per line):
   ├─ Short Packet (Frame Start): VC[1:0] | DT=0x00 | WC=0 | ECC
   ├─ Long Packet (Line): VC | DT | WC (byte count) | Data... | CRC16
   └─ Short Packet (Frame End): VC | DT=0x01 | WC=0 | ECC

4. EoT (End of Transmission)
   └─ HS-to-LP transition after each burst

Calculating HS settle time

HS settle time is the period the receiver waits after the LP-HS transition before sampling data. NVCSI uses a fixed settle time derived from the lane rate:

T_HS_SETTLE (ns) = (85 + 6 * UI)   [MIPI D-PHY spec minimum]
where UI = 1 / (2 * lane_rate_bps)

Example: 1.5Gbps lane rate
UI = 1 / (2 * 1.5e9) = 0.333 ns
T_HS_SETTLE = 85 + 6 * 0.333 = 87ns minimum

For Jetson NVCSI, add margin: use 100-150ns at 1.5Gbps

In the device tree, cil_settletime sets the NVCSI settle counter:

sensor_modes {
    mode0 {
        cil_settletime = "100";   /* nanoseconds; increase if HS lock fails */
        ...
    };
};

Start with a large value (e.g., 500) and reduce until lock fails to find the minimum margin.

Disabling deskew calibration for debug

During bring-up, disable deskew to isolate whether failures are calibration or data-format related:

/* In your camera or sensor node */
nvcsi@15a00000 {
    channel@0 {
        ports {
            port@0 {
                endpoint@0 {
                    /* Disable NVCSI deskew calibration */
                    nvidia,deskew-en = <0>;
                    ...
                };
            };
        };
    };
};

With deskew disabled, lane alignment relies on the FPGA transmitting clean, balanced differential signals. If the image works with deskew disabled but not with it enabled, your FPGA MIPI IP is either not transmitting the calibration sequence or transmitting it incorrectly.

Virtual channel configuration for multi-stream FPGA

FPGAs can multiplex multiple video streams on one MIPI link using virtual channels:

/* 3 virtual channels from one FPGA on CSI0 */
cam_module0 {
    badge = "fpga_vc0";
    position = "topleft";
    drivernode0 {
        pcl_id = "v4l2_sensor";
        sysfs-device-tree = "fpga@0";
        /* VC 0 */
    };
};

cam_module1 {
    badge = "fpga_vc1";
    position = "topright";
    drivernode0 {
        pcl_id = "v4l2_sensor";
        sysfs-device-tree = "fpga@1";
        /* VC 1 */
    };
};

The FPGA sets the VC field in each MIPI packet header; NVCSI routes VC0 → /dev/video0, VC1 → /dev/video1, etc.

Diagnosing deskew failures

# Check for deskew calibration failure
dmesg | grep -E "deskew|cphy|phy.*cal|hs.*settle|nvcsi" | head -20

# Common failure messages:
# nvcsi: deskew calibration failed on brick 0
# nvcsi: phy HS settle timeout on lane 0
# nvcsi: MIPI-CSI lane sync error

# Verify lane count and rate settings
v4l2-ctl -d /dev/video0 --list-formats-ext
# Check that format/resolution matches FPGA output

# Check NVCSI register state (kernel debug interface)
ls /sys/kernel/debug/tegra_nvcsi/
cat /sys/kernel/debug/tegra_nvcsi/status

Test pattern validation

Before connecting to your real FPGA image source, validate the pipeline with a fixed test pattern:

# On the FPGA side, output a color bar or solid color first
# This confirms the link works before debugging image content

# Expected dmesg on successful lock:
# nvcsi: deskew calibration complete brick 0
# vi: stream start vc=0

# Capture and view a frame
v4l2-ctl -d /dev/video0 \
  --set-fmt-video=width=896,height=896,pixelformat=RG10 \
  --stream-mmap --stream-count=1 --stream-to=/tmp/fpga.raw

# Convert and view
ffmpeg -f rawvideo -pixel_format bayer_rggb16le \
  -video_size 896x896 -i /tmp/fpga.raw fpga_frame.png

For the MIPI CSI-2 camera bring-up process with standard sensors (which uses the same NVCSI path), see MIPI CSI camera driver setup on Jetson Orin. For enabling the CSI interface in device tree including the VI node, see How to enable CSI0 on Jetson Orin.

FAQ

Can an FPGA be used as a MIPI CSI-2 source for Jetson Orin?

Yes. NVCSI accepts any CSI-2 compliant source. The FPGA must implement the full CSI-2 protocol including correct SoT/EoT sequences, packet headers, CRC, and deskew calibration for rates above 1.5Gbps.

What is nvcsi deskew calibration and why does it fail with FPGA sources?

Deskew calibration corrects lane-to-lane skew in the MIPI receiver. Camera sensors implement it automatically. FPGAs often skip or implement it incorrectly, causing NVCSI to fail at higher data rates.

What does ‘short frame’ mean in nvcsi context?

The MIPI Frame End packet arrived before the expected number of lines/pixels were delivered. The FPGA must output exactly the pixel count and line count specified in the device tree sensor mode.

What FPGA MIPI IP cores are compatible with Jetson Orin NVCSI?

Xilinx/AMD MIPI CSI-2 TX IP (PG232) and Intel/Altera MIPI CSI-2 IP both work. Ensure they generate deskew calibration sequences and meet HS settle timing requirements for your lane rate.

---