For years, adding AI to an app meant sending the user's content to a server. For many products that's fine. For anything personal, like notes, journals, messages or health information, it's a hard trade: the feature is helpful, but now someone else's computer has read your words.
That trade is getting easier to avoid. Modern phones and laptops run capable language and speech models locally, and platform makers now expose them to developers directly. It's a real opportunity, but on-device models are smaller than the largest cloud models, and designing well for them takes some care.
Pick tasks that fit the model
Smaller models are good at bounded tasks with the answer close at hand: give this entry a title, tidy up a transcript, pull out the to-dos, summarise this week in a few sentences. They're weaker at open-ended reasoning or at facts they'd have to remember from training.
So we design features that work with the user's own content rather than the model's general knowledge. The material is right there in the prompt, and the model's job is to reshape it, not to invent anything. That plays to its strengths and keeps its output grounded.
Treat model output as untrusted
A language model can be wrong with total confidence. We treat everything it produces the way we'd treat input from the internet: it gets checked before it's shown. A title that's empty, absurdly long or nothing like the entry is rejected. A summary that mentions a person or place that never appears in the source is a red flag.
We've learned to be careful with examples in prompts, too. Give a small model a concrete sample sentence to show the format you want, and it may copy the sample into real output. It's better to describe the shape you want in words and to check the result.
The original stays the source of truth. AI output sits beside it and never replaces it.
Keep the original, always
When AI rewrites something a person said or wrote, the original must survive untouched. A cleaned-up transcript sits beside the raw one, and the person can switch between them. A summary links back to what it summarises. If the model gets something wrong, nothing has been lost, and the person can see exactly where it went astray.
This is also the honest way to present AI: as a helpful reading of your material, not a replacement for it.
Describe, don't advise
There's a line we hold firmly in personal apps. AI features describe what's there: what you talked about, what you said you'd do, which topics come up often. They don't diagnose, judge or tell people how to live. A model that confidently offers advice about someone's health, relationships or feelings is overstepping, however kindly it's phrased.
Describing is genuinely useful, and it respects that the person, not the software, decides what their words mean.
Design for when it isn't available
On-device AI depends on the hardware, the operating system version, the language and sometimes a model download in the background. Any of those can be missing. A well-designed feature degrades gracefully: the app still records, saves and searches, and explains in plain words why a particular feature isn't available yet and what would change that.
Never quietly fall back to a server. If a private app sends data off the device only when the on-device path fails, it isn't a private app.
Know when to stay quiet
Some content shouldn't be summarised cheerfully or turned into a to-do. A thoughtful product lets its AI step back: no title suggestion, no recap, just the person's words, kept safely. Restraint is a feature.
Be precise about where processing happens
Finally, say exactly where the work happens, in the product and in the privacy policy. "Runs on your iPhone" is a claim people can hold you to, and it should be literally true. Vague phrases like "secure AI processing" invite suspicion. Specific ones build trust.
Eiderlane designs and builds apps and websites, including AI features that run on the device. Read more in the journal.