MQTT and Protobuf IPC for Edge AI Appliances

Most edge AI appliances end up running five or six independent programs on one board: a capture service reading the camera or sensor array, an inference service running the model, a connectivity service managing BLE or WiFi, a fusion service combining sensor readings, and an OTA or provisioning service handling updates. The question that decides whether that fleet stays maintainable is how those programs talk to each other. On appliances we’ve built, the answer is MQTT and protobuf IPC: a local broker carrying protobuf-encoded messages as the only communication path between services, with no direct sockets and no shared code. This post covers why that pattern holds up in the field, along with the container structure, logging strategy, and recovery layers that turn it from a demo into something you can leave running unattended for months.

Key Insights

• The MQTT broker is the only IPC surface: services never import each other’s code or open direct sockets to one another; if a message doesn’t cross the broker, it isn’t an interface between services
• Protobuf keeps the wire format compact and schema’d: services agree on message shape through a .proto contract instead of hand-rolled JSON parsing that drifts out of sync over time
• Per-service privileged containers trade isolation for hardware reach: acceptable on a single-tenant appliance where the container fleet is the entire system, not a pattern to carry over to shared infrastructure
• Logging is split by severity, not by service: verbose and debug logs live in a RAM-backed tmpfs and disappear on reboot; only error-and-above logs get written to flash, capped and rotated
• Recovery is layered, not left to Docker’s restart policy: an external process supervisor owns restart decisions for the whole fleet, and a hardware watchdog wired through an I2C GPIO expander is the last resort when userspace itself has wedged

MQTT and protobuf IPC as the only bus between services

The obvious alternative to a broker is direct connections: give the inference service a socket to the capture service, give the fusion service a socket to both, and so on. That works until the fourth or fifth service joins the fleet, at which point you’re maintaining N-squared point-to-point wiring, and adding a new service means touching every existing one that needs to talk to it. A local MQTT broker collapses that to one connection per service, always to the broker, regardless of fleet size. Adding a downloader or provisioning service later means subscribing it to the topics it cares about; nothing else changes.

We run eclipse-mosquitto as its own container, bound to 127.0.0.1:1883, with no cloud dependency for local traffic. The broker address is injected as an environment variable, MQTT_BROKER_ADDRESS, so the exact same service image runs against a remote broker on a developer’s laptop and the loopback broker on the device, unchanged. That single decision removes an entire class of “works on my machine, breaks on the board” bugs, because the IPC path is identical in both environments.

Protobuf as the payload format matters more than it looks. JSON would work fine if every service were written in the same language, but a fleet of services written in different languages needs a contract that isn’t tied to any one of their native serialization formats. A .proto schema compiles to typed bindings in each language, so a field rename or a new enum value is caught at build time in every service that consumes it, instead of surfacing as a silent KeyError on a device in the field six months later.

The other piece worth calling out is retained topics. Configuration and runtime state get published once with the MQTT retain flag set, and any service that subscribes afterward, whether it started a second later or after a container restart, immediately receives the last published value without waiting for the next update. That makes the broker the single source of truth for runtime config and removes the need for a separate config-distribution path (a shared file, a database, a sidecar). A service that restarts mid-flight doesn’t need special-case bootstrap logic; it just subscribes and gets caught up.

Structuring the container fleet

Each service gets its own container, one responsibility per container, sharing a common baseline set through a YAML anchor rather than duplicating settings across five service definitions:

Setting	Value	Why
restart	"no"	Docker doesn’t own recovery; an external supervisor does
network_mode	host	every container reaches the loopback broker with no bridge/NAT hop
logging.driver	json-file	standard driver, but capped (see logging section)
logging.options.max-size	1m	hard ceiling on flash writes per log file
logging.options.mode	non-blocking	log I/O never stalls the application
privileged	true (device-touching services only)	direct access to host devices and buses

The broker itself is the one service that stays unprivileged; it only needs the shared log mount, since it isn’t reaching into hardware.

If you’ve only ever run one container on a Genio or Jetson board (see our post on Docker with GPU acceleration on MediaTek Genio), the jump to a fleet mostly means deciding, per service, how far to open the door:

1. Memory caps on heavy services. A capture service holding frame buffers gets a deploy.resources.limits.memory ceiling in the low hundreds of MB, so a leak in one service can’t OOM the whole board.
2. Device passthrough scoped to what each service touches. An audio or voice service gets devices: [/dev/snd] plus read-only ALSA config mounts; it doesn’t get the whole /dev tree.
3. pid: host only where a service must see host processes. A host-side Bluetooth manager typically needs this; a pure compute service doesn’t.
4. Selective binary bind-mounts instead of a wholesale rootfs mount, where feasible. If a service only shells out to a handful of specific host binaries, mount those paths rather than the whole filesystem.

The two settings that make the rest of this possible, network_mode: host and a full /:/host_root bind-mount on privileged: true containers, are a deliberate trade. They give up per-container network isolation and most of the filesystem sandboxing Docker normally provides, in exchange for services being able to reach real host devices, firmware blobs, and tool binaries without baking a copy of each into every image. On a single-tenant appliance where the container fleet is the entire software stack, that’s a reasonable trade. On shared or multi-tenant infrastructure, it would not be.

Flash-wear-aware logging

Embedded flash (eMMC or SD) has a finite number of write cycles, and a chatty service fleet logging at debug level around the clock will wear through it faster than most teams expect, especially on a device that runs for years without a storage replacement. The fix is splitting log destinations by severity:

Log class	Destination	Survives reboot	Notes
Verbose / debug / info	/tmp (tmpfs, RAM-backed)	No	every container mounts the same RAM disk, so all services log to one place
Error and above	Docker json-file on flash	Yes	capped by max-size and max-file, non-blocking so log writes never stall the app

This changes how debugging happens day to day, for the better. Live tailing during development or a field debug session reads the RAM logs directly:

tail -F /tmp/<service>.log
tail -F /tmp/<service>.log | grep -i <event>

Post-mortem after a reboot or a crash only has the persisted error stream to work with:

docker logs <container>

The verbose logs are gone after a power cycle, on purpose. Keeping only error-and-above on flash is the entire point: you get full visibility while a device is up and reachable, and you get just enough on-flash history to diagnose why it went down, without writing gigabytes of debug chatter to storage that has to survive the product’s whole field life.

Recovery in layers: supervisor, then hardware watchdog

Every service container is configured with restart: "no". That’s deliberate: Docker’s built-in restart policies are per-container and don’t know about dependency order, backoff, or what “healthy” means for your specific fleet. Instead, an external process supervisor, a plain host-level process, owns start, stop, and restart decisions for the whole set. Centralizing that logic in one place means a single service crash-looping can be handled with backoff instead of Docker hammering it in a tight restart cycle, and a service that depends on another coming up first can wait for it, which Docker’s flat restart model has no concept of.

Above the supervisor sits a hardware watchdog, for the case the supervisor itself can’t help: a kernel panic, a wedged I2C bus, or the supervisor process itself hanging. The watchdog is kicked periodically through an I2C GPIO expander line. If the kick stops, because the supervisor or some critical service has wedged badly enough that nothing in userspace is running anymore, the watchdog forces a hardware reset. It’s not a replacement for the supervisor, it’s the layer beneath it: the recovery path for the failure mode where software-level recovery is no longer possible. The exact I2C address, expander part, and pin assignment are board-specific and need to be verified against your hardware before you rely on them; the pattern (a hardware timer, kicked by a live supervisor, resetting the board when the kick stops) is what carries over.

Getting this three-layer setup right (broker-mediated IPC, a supervisor with real restart policy, a watchdog underneath it) is most of what separates an appliance that self-heals from one that needs a truck roll every time a service wedges. If your fleet also has a service with a real-time constraint buried in it, a control loop inside the fusion service, for example, that’s a separate problem worth solving deliberately; see our post on PREEMPT_RT on Jetson Orin for what that involves.

Frequently Asked Questions

What is the best way to handle inter-process communication on an edge AI device?

For a fleet of independent services on one board (capture, inference, connectivity, fusion, OTA), a local MQTT broker carrying protobuf messages works well because it decouples every service from every other one. No service imports another’s code or opens a direct socket; the broker is the entire IPC surface. This matters more as the service count grows, since the alternative is N-squared point-to-point wiring.

Why use MQTT instead of gRPC or D-Bus for edge device IPC?

gRPC needs a client stub for every service pair, so five services means ten connections to wire and maintain. D-Bus is Linux-specific and gets awkward across container mount namespaces. A local MQTT broker gives every service one connection (to the broker) regardless of how many other services exist, and publish/subscribe means adding a new service does not require touching the ones already running.

How do you protect eMMC or SD card flash from wear in an embedded Linux appliance?

Split logging by severity. Route verbose and debug logs to a RAM-backed tmpfs mount that every container shares, and only write error-and-above logs to flash, capped with a max file size and rotation count. On a device running continuously for months, this is the difference between a log volume that never touches flash under normal operation and one that wears out the storage in weeks.

What is a hardware watchdog and do I need one for an edge AI appliance?

A hardware watchdog is a timer, often behind an I2C GPIO expander, that forces a hard reset if it stops being kicked. You need one any time software recovery might not be enough: a kernel panic, a wedged I2C bus, or a supervisor process that hangs. It’s the layer beneath your process supervisor, not a replacement for it, and it only matters at the point where userspace can no longer help itself.

Should each service run in its own Docker container on an embedded device?

Yes, when the services have different privilege and lifecycle needs. One container per responsibility (capture, inference, connectivity, fusion, provisioning) means you can restart, resource-cap, and update each independently, and a crash in one doesn’t take down the others. The tradeoff is that privileged, host-networked containers on a single-tenant appliance give up most container isolation in exchange for direct hardware access, which is a fair trade on a device that only ever runs your software.

---

ProventusNova helps hardware startups deploy production edge AI appliances fast. See our EdgeAI Deployment service.