Field notes
Offline-first inference: designing for the link that drops
Latency is the usual argument for edge inference. On farms, solar arrays and in orbit the real reason is that there is no uplink. How to design for that.
Latency is not why most industrial AI runs at the edge
The standard argument for edge inference is latency. Round-tripping to a cloud costs tens of milliseconds, a production line cannot wait, therefore inference belongs on the device. It is a real argument and it applies to a minority of deployments.
Walk around the places this software actually lives and a different reason dominates. A barn on a farm with one contended rural link. A photovoltaic array on a hillside, where the nearest connectivity is whatever the site's telemetry uses. A nanosatellite with a downlink window measured in minutes per orbit. In all three, the question is not how fast the round trip is. It is whether there is a round trip at all.
That distinction matters because it changes what you build. Designing for latency gets you a fast local model. Designing for absent connectivity gets you a system that keeps behaving sensibly for three days while nobody can reach it, which is a different and harder piece of engineering.
Four degraded modes, and choosing one on purpose
When the link goes, a system does one of four things. The failure is not choosing badly. The failure is not choosing, and finding out which one you built during the first outage.
- Buffer and forward. Keep inferring, store results locally, send when the link returns. Correct when the output is a record rather than an instruction. Needs a bounded store and a policy for what to discard when it fills.
- Act locally, reconcile later. The device makes and executes decisions alone, and the central system learns about them afterwards. Correct when the action is time-critical and reversible. Needs conflict resolution for when central state has moved on.
- Degrade to a simpler rule. Fall back to a threshold or a heuristic that does not need the full pipeline. Correct when partial function beats none, and honest about being worse.
- Stop and announce. Refuse to act, raise a local alarm. Correct when a wrong decision is more expensive than no decision, which in safety-adjacent contexts it usually is.
Only the fourth is safe by default, and it is the one that annoys operators most, so it tends to get replaced quietly by the first without anyone writing down that the decision changed.
What an offline-first deployment has to answer
- How long can it run unreachable? A number in hours or days, tested by disconnecting it, not estimated.
- What fills first? Local storage, a queue, a log. What gets dropped when it is full, and who decided that.
- How does a model get down the link? Resumable transfer, integrity check, and a device that keeps running the old model until the new one is verified.
- How do you know what version is where? A fleet with unknown model versions cannot be debugged, and after two years it will have at least three.
- What is the rollback? A bad model reaching forty unreachable devices is the worst day in this line of work. Keep the previous version on the device.
- How does the device prove it is alive? Absence of data must be distinguishable from absence of events. A silent camera and a camera seeing nothing look identical otherwise.
- What does the clock do? Offline devices drift. Timestamps that cannot be ordered across a fleet make the data much less useful later.
Shipping a model down a link that keeps dying
Model artefacts are large, links in the field are thin and intermittent, and an interrupted transfer that leaves a half-written file on the device is a failure mode that will find you. Three properties make this survivable: transfers resume rather than restart, the artefact is verified before it is activated, and activation is atomic so a device is always running either the old model or the new one and never a mixture.
None of this is novel. It is ordinary software distribution discipline, and it is routinely skipped in machine learning projects because the pilot updated one device by hand and nobody revisited the question.
When the model on the device is out of date
An offline device will sometimes run a model that central knows is superseded. That is tolerable and normal. What is not tolerable is nobody being able to tell, afterwards, which model produced which output.
Stamp every result with the model version that produced it. It costs a few bytes and it is the difference between "accuracy dropped in March" and "accuracy dropped in March on the eleven devices still on v4, and here they are".
Buffer and forward
- Central record stays complete once the link returns
- Simple to reason about and to test
- No divergent state to reconcile
- Right when the output is evidence, not an instruction
Act locally and reconcile
- The system keeps doing useful work while isolated
- Needs conflict resolution nobody enjoys writing
- Actions taken on stale information cannot be undone
- Right when waiting is itself the expensive outcome
My take
"Edge" is mostly discussed as a latency story, and I think that framing comes from people whose worst connectivity experience is a slow data centre. In the field it is an availability story, and the two lead to different architectures.
The tell is what a system does when you unplug the network. If the answer is a shrug and a stack trace in a log nobody reads, it was designed as a cloud system that happens to run locally. If the answer is a defined degraded mode, a bounded buffer and an alarm somebody receives, it was designed for the field. Almost every edge AI system I have looked at has been the first kind, including several sold specifically as edge platforms.
My practical advice is to make the outage a scheduled test rather than an incident. Pull the link during the pilot, on purpose, on a Tuesday morning, with everyone watching. You will find three things wrong, all of them cheap to fix at that point and expensive to fix once there are forty devices in four countries. Nobody ever regrets having run that test. Plenty of people regret not having.
This is the constraint behind most of our field work. We have since published the numbers from one such deployment: a 99% reduction against raw video, events small enough for a 200-byte satellite payload, and full data integrity through field testing at under 10% connectivity uptime. If you are planning a deployment where the link is unreliable, the pilot design matters more than the model, and the hardware question comes later than you think. To talk through a specific site, book a discovery call.