Building a Tablet Cockpit for a Robot

The dashboard that drives our robot has to run on whatever tablet is on hand, which is a harder requirement than it sounds. Two days before a show, it froze on the demo tablet in the most confusing way possible: the live camera worked and nothing else did. The cause is a great lesson in building interfaces for devices you do not control.

Key Insights

• The robot serves its own installable web dashboard: a first-person camera view, a joystick, a live lidar radar, and voice, with no cloud
• On one tablet the page froze with the camera live but every button and readout dead
• The cause was a single unsupported browser call early in startup that threw and halted everything scheduled after it
• Parts set up before the failing call kept working, which is why the symptom looked so strange
• A field-ready UI guards modern browser features, wraps its draw loops, and limits live connections so one failure cannot take the page down

What the cockpit is

We built the robot’s operator interface as a web app the robot serves itself. Any tablet or phone on the robot’s network loads it, and it can be installed to the home screen so it feels like a native app. It shows a first-person camera view with a joystick for driving, a live radar scope drawn from the lidar so the operator can see obstacles around the robot, and voice controls. Serving it from the device means it needs no app store and no internet, only the robot.

The freeze

At setup, the dashboard came up on the demo tablet with a working camera feed and nothing else. The drive joystick did nothing, the telemetry never updated, the command buttons were dead. It was not the network, because a laptop on the same connection showed everything working. It was not the camera streams competing for bandwidth, because the heaviest thing on the page, the video, was the part that worked.

The way we found it was to log what the tablet actually requested. It asked for the page and the camera feed, and it never asked for telemetry and never sent a single button press. That pointed straight at the cause: the page’s startup script had thrown an error partway through, so everything it tried to set up after that point simply never happened.

The cause

The lidar radar drew its sweep using a modern browser drawing call that the tablet’s older browser did not support. The first time the page tried to draw the radar, that call threw. Because it threw during the page’s initialization, everything scheduled after it, the telemetry stream, the live status, the buttons, the joystick, never got wired up. Everything before it, the camera feeds and a health check, had already been set up and kept working. That is why the symptom was so lopsided: a live camera on top of a completely dead control panel.

The lesson: build for devices you do not control

The fix was to stop assuming the browser. Guard the modern drawing call before using it so an unsupported feature is skipped rather than fatal, wrap the drawing loop so a failure inside it cannot halt the rest of the page, and be economical with live connections, because a thin wireless link and an older browser both have hard limits on how many simultaneous streams they will hold. With those in place the dashboard degrades gracefully instead of dying, and it works on whatever tablet is at the booth.

The general principle is the one in the heading. A dashboard that only runs on your development machine is not done. A field UI has to survive an unknown, possibly old browser on an unknown network, and that means defensive initialization so that no single unsupported feature can take the whole interface down.

We keep the implementation details for the projects we do, because they depend on your stack. The reusable insight is to treat the operator’s device as hostile terrain and build the UI to tolerate it.

If you are building operator interfaces for robots or edge devices that have to run anywhere, tell us what you are building. For the robot this drives, see the ROSOrin build overview.