NPU vs CPU on the MediaTek Genio: the Thermal Reality

Here is a lesson that surprises product teams: on a small fanless edge board, running AI is a heat problem before it is a speed problem. We learned it the direct way, by watching our robot’s MediaTek Genio climb past its throttle temperature within a minute of turning the camera on. This is what that looks like and how to think about it.

Key Insights

• Running our vision model on the MediaTek Genio 520 CPU drove the chip to 78 to 84 C and past its passive throttle point in about a minute
• The same model on the NPU held 58 to 60 C on the same board, cool enough to run fanless
• Capping CPU usage does not solve it: if inference is already too slow, throttling only makes it slower
• Some embedded kernels do not expose a working CPU quota control at all, so that lever may not even exist
• On a fanless edge device, plan the thermal budget of your AI workload before integration, not after

What overheating looks like

We ran our object detector on the Genio’s CPU cores first, because that is the path of least resistance. Within about sixty seconds of real camera frames, the board was at 78 to 84 C. The passive design wants to throttle well below that, so the chip started slowing itself down to protect itself. Every core was pinned at maximum clock the whole time, drawing six to nearly eight cores’ worth of load for a detector running at roughly two frames a second.

A throttling chip is the worst of both worlds. You are already too slow, and now the hardware makes you slower to keep from cooking. On a robot with no fan, in a warm room, this is not a demo you can leave running, let alone a product you can ship.

Why you cannot throttle your way out

The instinct is to cap the CPU so it runs cooler. It does not work here, for two reasons. First, the model is already slower than the frame rate we want, so limiting cores just trades heat for even lower frame rate, and you still may not get under the thermal limit. Second, and this is the embedded twist, the kernel on this board did not expose a working CPU quota control at all, so the usual container-level cap was simply unavailable. The lever you reach for may not be there.

The workload is on the wrong processor. That is the actual problem, and cooling tricks only paper over it.

The real answer

Moving the detection onto the Genio’s neural accelerator dropped the board to 58 to 60 C doing the same job, cool enough to run without a fan while the camera and the rest of the stack ran alongside it. That is the headline: the NPU is not only faster, it is what makes on-device vision thermally possible on a passive board. We cover the speed side in YOLOv8 on the Genio NPU.

If for some reason a workload has to stay on the CPU, then the honest options are a smaller, cheaper model and real cooling, a heatsink or a fan sized to the sustained load. Both are legitimate, and both are decisions you want to make early.

The order of operations that saves months

The expensive mistake is discovering your thermal problem at integration time. The order that avoids it is: choose the board for its accelerator, confirm your model runs on that accelerator within your frame-rate and temperature budget, then design the enclosure and cooling around the load you measured. Teams that skip the measurement and assume the CPU will cope tend to find out in the enclosure, which is the most expensive place to find out.

Sizing an AI workload to a fanless board, on the MediaTek Genio or on NVIDIA Jetson, is exactly the kind of thing we characterize quickly because we have done it before. Tell us what you are building and we will tell you whether your model fits your board before you commit to the enclosure. For the robot this came from, see the ROSOrin build overview.