Hardware-accelerated GStreamer live streaming on Jetson Orin

Building a hardware-accelerated live streaming pipeline on Jetson Orin means using nvv4l2h264enc — the Jetson NVENC engine — instead of software encoders. The difference is significant: software H.264 encoding (avenc_h264) can barely sustain 1080p30 on Jetson’s Cortex-A78 cores, while nvv4l2h264enc handles it at under 5% CPU load. This post covers RTSP server setup, UDP streaming, latency tuning, and how to receive the stream on any device.

How to create a hardware-accelerated GStreamer pipeline for live streaming on Jetson

A hardware-accelerated GStreamer live streaming pipeline on Jetson Orin works as follows: nvarguscamerasrc captures from the CSI camera through the Jetson ISP in NVMM memory, nvvidconv converts the format within NVMM, nvv4l2h264enc encodes in the NVENC hardware engine, h264parse prepares the bytestream, rtph264pay wraps in RTP, and udpsink or an RTSP server delivers the stream.

The key constraint is keeping everything in NVMM memory until the encoded output — inserting any CPU element (videoconvert, appsink) before the encoder forces a copy from GPU memory to system RAM, adding latency and CPU overhead.

Key Insights

• nvv4l2h264enc is the only correct choice for live streaming on Jetson — it encodes entirely in NVENC hardware with near-zero CPU usage
• Keep all elements in NVMM between nvarguscamerasrc and nvv4l2h264enc — a single videoconvert in this path copies every frame through system RAM
• preset-level=1 (UltraFast) + tune-level=1 (ZeroLatency) on nvv4l2h264enc cuts encoder latency by 60-70% vs default settings
• sync=false on all sinks removes the presentation timestamp delay that adds 33ms+ of artificial latency
• For RTSP, gst-rtsp-server with a nvv4l2h264enc launch string handles multi-client streaming without running one encoder per client

UDP streaming pipeline (fastest setup)

UDP is the simplest live streaming setup — no server process needed, one receiver, lowest overhead.

On Jetson (sender):

gst-launch-1.0 \
  nvarguscamerasrc sensor-id=0 ! \
  'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12' ! \
  nvv4l2h264enc preset-level=1 tune-level=1 bitrate=4000000 iframeinterval=30 insert-vui=1 ! \
  h264parse ! \
  rtph264pay pt=96 config-interval=1 ! \
  udpsink host=<RECEIVER_IP> port=5000 sync=false

On the receiving machine:

gst-launch-1.0 \
  udpsrc port=5000 ! \
  'application/x-rtp,media=video,payload=96,clock-rate=90000,encoding-name=H264' ! \
  rtph264depay ! \
  avdec_h264 ! \
  autovideosink sync=false

Replace <RECEIVER_IP> with the laptop or server IP. Use ifconfig or ip addr on the Jetson to find its IP.

nvv4l2h264enc key properties:

Property	Value	Effect
preset-level	1 (UltraFast)	Fastest encode, some quality trade-off
tune-level	1 (ZeroLatency)	Disables B-frames and look-ahead; adds no buffering delay
bitrate	4000000	4 Mbps for 1080p30 — adjust up for quality
iframeinterval	30	I-frame every 30 frames (1 second at 30fps)
insert-vui	1	Embeds VUI timing info; helps receivers sync
insert-sps-pps	1	Embeds SPS/PPS in every IDR; required for late joiners

RTSP server with gst-rtsp-server

RTSP allows multiple clients to connect and reconnect without restarting the pipeline. Install the server library:

sudo apt install python3-gi gir1.2-gst-rtsp-server-1.0

Python RTSP server — save as rtsp_server.py:

import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstRtspServer', '1.0')
from gi.repository import Gst, GstRtspServer, GLib

Gst.init(None)

server = GstRtspServer.RTSPServer.new()
server.set_service("8554")

mount_points = server.get_mount_points()

factory = GstRtspServer.RTSPMediaFactory.new()
factory.set_launch(
    "( nvarguscamerasrc sensor-id=0 ! "
    "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12 ! "
    "nvv4l2h264enc preset-level=1 tune-level=1 bitrate=4000000 iframeinterval=30 ! "
    "h264parse ! "
    "rtph264pay name=pay0 pt=96 config-interval=1 )"
)
factory.set_shared(True)  # one pipeline, multiple clients

mount_points.add_factory("/live", factory)
server.attach(None)

print("RTSP server running at rtsp://0.0.0.0:8554/live")
GLib.MainLoop().run()

Run it:

python3 rtsp_server.py

Connect from VLC, ffplay, or another Jetson:

# VLC
vlc rtsp://<JETSON_IP>:8554/live

# ffplay (low latency flags)
ffplay -fflags nobuffer -flags low_delay -framedrop rtsp://<JETSON_IP>:8554/live

# Another GStreamer pipeline
gst-launch-1.0 rtspsrc location=rtsp://<JETSON_IP>:8554/live ! decodebin ! autovideosink sync=false

factory.set_shared(True) means all clients see the same encoded stream — the encoder runs once regardless of how many clients are connected. Without this, each client would start its own pipeline and camera session.

Using v4l2src instead of nvarguscamerasrc

If your camera is a USB camera or a sensor not in the Argus registry, use v4l2src with explicit format caps:

gst-launch-1.0 \
  v4l2src device=/dev/video0 ! \
  'video/x-raw,format=YUY2,width=1280,height=720,framerate=30/1' ! \
  nvvidconv ! \
  'video/x-raw(memory:NVMM),format=NV12' ! \
  nvv4l2h264enc preset-level=1 tune-level=1 bitrate=2000000 iframeinterval=30 ! \
  h264parse ! \
  rtph264pay pt=96 config-interval=1 ! \
  udpsink host=<RECEIVER_IP> port=5000 sync=false

The nvvidconv element converts v4l2src system memory output into NVMM for nvv4l2h264enc. Caps conversion through nvvidconv is required — nvv4l2h264enc does not accept system memory buffers directly.

H.265 encoding for lower bandwidth

For bandwidth-constrained links, H.265 (HEVC) cuts bitrate roughly in half at the same quality level. Replace nvv4l2h264enc + h264parse + rtph264pay with:

nvv4l2h265enc preset-level=1 tune-level=1 bitrate=2000000 iframeinterval=30 ! \
h265parse ! \
rtph265pay pt=96 config-interval=1 ! \
udpsink host=<RECEIVER_IP> port=5000 sync=false

Receive on the other side:

gst-launch-1.0 \
  udpsrc port=5000 ! \
  'application/x-rtp,media=video,payload=96,clock-rate=90000,encoding-name=H265' ! \
  rtph265depay ! \
  avdec_h265 ! \
  autovideosink sync=false

H.265 decode is more CPU-intensive on the receiver. If the receiver is also a Jetson, use nvv4l2decoder instead of avdec_h265.

Latency tuning

With default settings, a Jetson live stream often has 200-500ms of latency. These changes bring it under 100ms:

Encoder side (Jetson):

• preset-level=1 — eliminates encoder look-ahead buffering
• tune-level=1 — disables B-frames; each frame is independently decodable immediately
• iframeinterval=30 — lower values add overhead; 30 is a good balance for fast recovery

RTP/network side:

• config-interval=1 on rtph264pay — embeds SPS/PPS every second; receivers don’t wait for the first IDR
• udpsink sync=false — no presentation timestamp waiting

Receiver side:

• udpsrc / rtspsrc typically add 50-200ms of jitter buffering by default
• For rtspsrc: set latency=50 property (milliseconds) — rtspsrc latency=50 location=rtsp://...
• For ffplay: add -fflags nobuffer -flags low_delay -framedrop

Measuring actual end-to-end latency

Display a millisecond clock in the camera frame and film both the Jetson display and the receiver screen simultaneously to measure actual end-to-end delay. Alternatively, use GStreamer’s GST_DEBUG_DUMP_DOT_DIR to see buffer timestamps, or add a fakesink with dump=true to inspect buffer PTS values at the output.

# Quick latency check pipeline — no network, renders to screen
gst-launch-1.0 \
  nvarguscamerasrc sensor-id=0 ! \
  'video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1' ! \
  nvv4l2h264enc preset-level=1 tune-level=1 bitrate=4000000 ! \
  h264parse ! \
  nvv4l2decoder ! \
  nvvidconv ! nv3dsink sync=false

This encode-then-decode loop on the same device gives a rough encoder+decoder latency figure. On Jetson AGX Orin at 1080p30 with UltraFast + ZeroLatency, this pipeline runs at under 30ms round-trip.

Debugging common issues

nvv4l2h264enc: Error while encoding — almost always an NVMM caps mismatch. Verify nvvidconv is outputting video/x-raw(memory:NVMM),format=NV12 before the encoder.

Stream plays but freezes every few seconds — IDR interval too large or receiver jitter buffer overflow. Lower iframeinterval to 15 and add insert-sps-pps=1.

RTSP clients connect but see black video — factory launch string syntax error. Test the launch string standalone with gst-launch-1.0 first, then wrap in parentheses for the factory.

High CPU on Jetson during streaming — a CPU-based element crept into the pipeline. Run htop while streaming and check which processes are consuming CPU. Add GST_DEBUG=nvvidconv:3 to trace NVMM buffer flows.