Remote Firmware Updates Over the Mesh
The hard part of a statewide mesh is not flashing a radio on a bench. It is changing firmware on a repeater you will not see again for months, over a link that is already busy carrying real traffic, without bricking the node or stepping on the mesh.
That is what mota is for. A .mota file is a self-checking firmware package that can travel over LoRa a kilobyte at a time. Neighbors advertise what they have. A node fetches the pieces it needs at the lowest radio priority we have. Nothing installs until someone says so, or until you have explicitly trusted a signer.
The original OTA design is vk496's. EnvyOS packages it with motatool, an apply bootloader we call EnvyBoot, and a release habit that matters more than any of the file formats: one delta from every prior fleet image, straight to latest.
What belongs on the mesh
There is a lot of talk about what data should be allowed on the mesh and how often it should be allowed. Some people feel that human-generated data is the only thing that should cross the air. I do not share that view. The mesh is a public resource in the ISM band. The nature of the data a user transmits is nobody else's business. The frequency of that transmission is worth more of a conversation.
As a staunch privacy advocate, I make no distinction between data generated or destined for a computer versus data for a human. It is all ultimately for humans to experiment with as they see fit. I am also a staunch advocate for sovereignty. If a repeater operator does not want certain types of data, or certain senders, passing through their repeater, that is equally their prerogative. None of us are obligated to forward data for anybody else. None of us are bound by anyone else's opinion of what data or what transmission rates are acceptable.
Part of the issue in the early evolution of the mesh is that we do not have many controls for shaping and managing traffic so that only desired traffic passes through repeaters. When I say we, I mean everybody in the community, not MeshEnvy alone. MeshEnvy's personal stance is to forward everything and filter nothing, and we will continue doing that until it creates real, observable, actionable problems. It is a great philosophical debate. I land on one side of it. Reasonable minds differ and land on another. That is the great thing about the mesh. Repeater sovereignty means we can each do what we think is best for the future of the network.
Remote firmware updates sit right in the middle of that debate. I think being able to update repeater firmware over the mesh is critically important for the MeshEnvy project. It is high on my list of problems to solve in a way that has a good-neighbor effect while keeping the mesh healthy, secure, and available for everyone.
For day-to-day work we run bench seeders at 1% to 10% duty cycle so updates trickle quietly. Hypothetically, a statewide emergency might need a rapid firmware response. Raise the duty cycle and you could propagate an upgrade across the state in minutes. That would cause temporary congestion. It also makes it possible to fix a critical error or respond to infrastructure failure in near real time. I think that is a powerful enough use case to show there is no one clear answer to what kind of data should flow, from whom, or how quickly. Bad actors exist. "Bad" is also in the eye of the beholder sometimes. Would a rapid response to a critical emergency be bad? Not necessarily. There is room for debate. The essence of remote firmware update is clearly valuable and worth experimentation.
I am not the pioneer here. Others who came before me have done mind-blowing work and continue to. I got an email the other day from someone working on even better compression strategies. Several community members are exploring techniques like two repeaters handshaking, switching frequencies off the main band for a bulk transfer, and then coming back online. I can see that working in some cases. In others, if you are the only repeater that can route through an area, going offline for a bulk transfer would not make sense. In-band transfer would. Again, there is no one right solution. Any experimentation and research in this area has tremendous value.
We run a custom MeshCore distribution, EnvyOS, that overlays mota and various smaller bug fixes and improvements. Everybody is welcome to check out that open source project and decide for themselves whether they want their repeater to participate in the over-the-air update network. The more participants, the less bandwidth and airtime it ultimately takes any one node to propagate an update.
I do not know whether our solution will be the final one, or if there even will be a final one. It is not clear to me that this feature should be a core feature at all. It has high potential, at least in the present day, for misconfiguration and for bricking devices with no option but to roll a truck to fix them. At this point it is fragile and needs to be handled with care by someone who understands the process. I do a lot of bench testing before I am confident doing a remote update I might have to get in my Jeep to go fix. I wanted to share more technical background because the topic is genuinely interesting to me.
The old pain
USB on a summit works until it doesn't. Weather turns. A site is a two-hour climb. The person who can get there next is a volunteer with a Saturday, not a technician on call. In winter some peaks simply do not get summited at all.
Nevada brings a particular perspective to this problem. We are one of the most mountainous states there are. It is really attractive to put nodes on peaks, and really hard to maintain them. We have vast open space and a long list of ridge sites that take hours to reach. I think that puts us in a somewhat unique position to care about remote firmware updates in a tangible way. I would not expect the same urgency in a dense urban mesh where everyone hangs a repeater out a window and a USB cable is right there. I do not know that contrast is universally true. It is just what it looks like from here. Other communities may not contend with frozen summits and multi-hour approaches the same way we do.
Even when you can reach the node over the air, a full firmware image is the wrong object to send. Our slim repeater builds are around 400 KiB as a full .mota. On Medium Fast, that is a lot of airtime. On a busy backbone it is enough airtime that the transfer never quite finishes.
Deltas change the math. A nearby EnvyOS bump is often 7 to 15 KiB. A jump from the original v0.1.0 fleet image to v0.1.2 is about 41 KiB on the slim RAK4631 repeater. When we rebased onto MeshCore companion-v1.17 for EnvyOS 0.2.0, the delta from that same first fleet image was still under 100 KiB.
Those are the sizes that make "update it from the valley" a real sentence.
What a .mota actually is
A .mota is not a raw hex file with a new extension. It is a container with a manifest, a merkle tree, and a payload.
The payload is either the whole firmware image or a detools patch. The patch is the interesting case. It is a compressed list of edits against a specific base image. The node already has that base. It is the firmware it is running.
Every EnvyOS build stamps a short trailer on the image called EndF. The trailer carries a hash of the firmware body, a version, and a target id (board plus role). When we build a delta, motatool writes that body's hash into the manifest as base_hash. At apply time the node compares base_hash to its own EndF. If they do not match, the patch is refused. A delta for a different build will not "kind of" apply.
The target id is the other brick-safety check. A WisMesh Tag companion image will not look like [yours] on a RAK4631 slim repeater. The node can see the file. It will not treat it as something to install.
Discover, fetch, then maybe install
The loop is three steps, and they stay separate on purpose.
Discover. Nodes advertise the .mota files they can serve. ota ls is the human view. You see version, full versus delta, how many neighbors have it, and whether it is built for this hardware and role.
Fetch. ota get pulls the file a block at a time, default 1024 bytes, with a merkle proof on each block. The node can take blocks from more than one neighbor. Progress lives in flash, so a reboot or a dead afternoon of traffic does not throw the work away. The next quiet window fills the holes.
Install. ota install is a separate command. The node verifies the merkle tree, the image hash, the base hash, the hardware tag, and an optional Ed25519 signature. On nRF52 it then writes a local approval mark and reboots into EnvyBoot. The running app never patches itself. The bootloader applies the in-place patch and will not boot the result unless it hashes to the manifest's image_hash.
If you never type ota install, the file just sits there. Auto-install exists, but it is off unless you turn on ota config autoinstall trusted and add a signing key. That is the default I want on a public mesh.
Why this is safe to run on a live mesh
The first safety property is radio, not crypto.
OTA packets use the lowest transmit priority in MeshCore. Messages, routing, and admin traffic go first. A busy node delays the update. We describe that as eventually upgradable, and I mean it literally. The mesh does not owe you a firmware transfer this hour.
The second property is that relays do not have to be trusted. Any node can carry a block. Integrity is content-addressed against a merkle root in the manifest. A neighbor can be wrong, stale, or hostile. A bad block fails its proof and gets fetched again. The relay never needs the signing key.
The third property is consent. Discovery and download are not install. A peer cannot pre-approve a container. The approval word is forced erased on ingest and only flipped locally.
The fourth is recovery. A failed nRF52 apply lands in UF2 recovery, not a half-written app. ESP32 keeps the previous slot. USB flash still works when all else fails.
Taken together, that is why I am willing to let OTA share a channel with people who are trying to send a message. The update loses every argument about airtime, and it cannot install itself by accident.
One hop from any base to latest
Most firmware systems think in chains. You are on N. The next patch is N to N+1. Miss a release and you either replay the whole chain or you pull a full image.
We do not do that.
When we cut a release, the build emits a full .mota plus one delta from every prior released image for that target. v0.2.0 shipped delta-from-v0.1.0, delta-from-v0.1.1, and delta-from-v0.1.2. A node on the original fleet image jumps to latest in a single fetch. A node that already took 0.1.2 does the same, with a smaller patch.
The running firmware is the base. Latest is the only destination. There is no "you must be on yesterday's build first."
That is the whole strategy. It costs us a few extra motatool jobs at release time. It saves the mesh from chained transfers and from the full-image fallback that our slimmer repeaters often cannot stage anyway. A full slim .mota is larger than the leftover flash those units have for staging. Deltas are not only faster. On those boards they are the path that fits.
How small is small
Published v0.1.2 numbers, from the GitHub release:
| Target | Full | From v0.1.1 | From v0.1.0 |
|---|---|---|---|
| RAK4631 slim repeater | 385 KiB | 7 KiB | 41 KiB |
| WisMesh Tag repeater | 491 KiB | 8 KiB | 46 KiB |
| WisMesh Tag companion | 459 KiB | 15 KiB | 39 KiB |
Adjacent EnvyOS bumps land in the single-digit to low-teens KiB. A two-version jump stays around 40 KiB. The 0.2.0 rebase onto a new MeshCore companion tag is the stress case, and even that stayed under 100 KiB from the first fleet image.
I treat about 10 to 50 KiB as the normal over-the-air object, and about 400 KiB as the thing we try not to send. We are bench-testing superseeders that would cache deltas on SD or QSPI for the same reason. That path is not released yet. The full snapshots stay on GitHub and on a laptop for USB bootstrap.
Once any node finishes a fetch, it offers that file to its neighbors. The update spreads outward instead of everyone hammering the first seeder. Lowest priority still applies, so the epidemic stays polite.
What I am still chewing on
Superseeders are the field path I want: a RAK4631 with a microSD, or a SenseCAP P1-Pro with its 2 MB QSPI, sitting on a ridgeline with a folder of deltas and no USB cable. Tethering a laptop to a seeder node is a chronic failure point. I wrote about that direction in Introducing EnvyOS. Today the bench still uses motatool serve because it is how we pack and verify. Superseeders exist only in bench testing so far. They have not shipped.
Bootloader updates are the other half of full upgradeability. Firmware and bootloader have a symbiotic relationship in our stack. Firmware can replace the bootloader. The bootloader can replace firmware. Right now a .mota moves application firmware through EnvyBoot, and bootloader changes still tend to mean a USB window when someone is already on site. Bootloader updates seem infrequent to me, but for completeness they matter. The goal is the same as firmware OTA: fix something critical without climbing the mountain. We plan to extend the mota path so bootloader rolls can travel over the mesh too. Watchdog coverage in EnvyOS 0.2.0 is the kind of fix where the firmware delta can arrive over LoRa while the bootloader piece still waits for hands on the radio.
If you run EnvyOS and want the operator view, the device side is ota help. If you want the wire format, it lives in the OTA protocol spec on the envycore tree. This post is the why.