Work
On-device inference, with nothing leaving the device
A working prototype: speech and language models that run entirely on the device, with no network call at inference time, for settings where connectivity is unreliable or data cannot leave the hardware.
- What it proves
- Privacy by architecture: the model runs where the data is, so nothing is sent anywhere.
- Method stages
- Built from
- On-device speech recognition
- On-device language model inference
- No network round trip at inference time
- No data leaving device hardware
- Sectors
The problem
Most AI privacy claims are policy claims: the vendor will not train on your data, will delete it after a set period, and has an agreement that says so. You cannot verify any of that. Connectivity is a second problem: a factory basement, a remote industrial site or a clinical area with restricted network access may have no reliable link at all.
What we built
Speech recognition and language-model inference both run on the device, with no server round trip at any point. There is no cloud step to secure, trust or explain to an auditor. The system also works with no connection.
Model size, quantisation and latency are engineered against the hardware that exists. We design for on-device from the first day, because a cloud system shrunk afterwards tends to work like a degraded cloud system.
What it proves for a client
Privacy can be a property of where computation happens. Anyone can check it by confirming that no network call was made. The same approach runs in our own voice-first app, Zeza.
Method stages
It belongs to Stage 02 · Prove the value, where a prototype tests whether the constraint is workable on your hardware. Stage 03 · Engineer the system would harden it for a specific device fleet. See the KARPINITY Method.