What On-Device AI Means
On-device AI runs a model on the phone's own CPU, GPU or neural engine instead of sending data to a server. Both platforms now ship a system-managed language model that apps can call without bundling any weights: Apple's Foundation Models framework and Android's Gemini Nano, exposed through ML Kit GenAI APIs on top of the AICore system service.
Apple: Foundation Models Framework
Foundation Models (iOS, iPadOS, macOS and visionOS 26+) gives Swift code direct access to the on-device model behind Apple Intelligence. It only works on Apple Intelligence-capable devices (on iPhone, iPhone 15 Pro and later), with Apple Intelligence turned on and the model downloaded.
Check availability first
switch SystemLanguageModel.default.availability {
case .available:
showSmartFeatures()
case .unavailable(.deviceNotEligible):
showClassicUI()
case .unavailable(.appleIntelligenceNotEnabled), .unavailable(.modelNotReady):
showEnableHint()
case .unavailable:
showClassicUI()
}
Guided generation and tool calling
Instead of parsing free text, mark a type with @Generable and let the framework guarantee a valid instance:
@Generable
struct TripIdea {
@Guide(description: "Short title, max 6 words")
let title: String
@Guide(description: "Trip length in days", .range(1...7))
let days: Int
}
let session = LanguageModelSession(instructions: "You plan short trips.")
let idea = try await session.respond(to: "Weekend near Izmir",
generating: TripIdea.self).content
Tools conform to the Tool protocol (name, description, @Generable Arguments, call(arguments:)). The model decides when to call them, so it can query your local database or app services.
Limits to plan for
- Context is small: 4,096 tokens on the original iOS 26 model. From iOS 26.4, read
contextSizeand usetokenCount(for:)instead of hard-coding. - A session handles one request at a time; check
isResponding. - Apple's own docs say the model is not suited for math, code generation or complex reasoning. It shines at summarization, extraction, classification and short creative text.
2026 additions
WWDC26 added image attachments in prompts, PrivateCloudComputeLanguageModel (a larger server model that requires an entitlement), a LanguageModel protocol for swapping models behind the same session API, and watchOS 27 support. iOS 27 also introduced Core AI for running your own neural models on Apple silicon; Core ML remains the documented choice for non-neural models such as decision trees, and the only option on iOS 26 and earlier.
Android: Gemini Nano via ML Kit GenAI
AICore downloads, updates and shares one Gemini Nano base model across apps. As of October 2026, ML Kit offers:
- Feature-specific APIs (beta): summarization, proofreading, rewriting and image description, each fine-tuned with a built-in prompt.
- Direct model access (beta): first announced as the Prompt API, now the
GenerativeModelclass for custom text or image-plus-text prompts. Structured output and function calling are still alpha/preview.
Device support is limited to recent flagships such as the Pixel 9, 10 and 11 series, Galaxy S25 and S26, and selected Honor, OnePlus, OPPO, vivo, Xiaomi and Motorola models. Direct model access depends on the Gemini Nano version (nano-v2, v3, v4), and output can differ between versions.
val summarizer = Summarization.getClient(
SummarizerOptions.builder(context)
.setInputType(InputType.ARTICLE)
.setOutputType(OutputType.ONE_BULLET)
.setLanguage(Language.ENGLISH)
.build()
)
when (summarizer.checkFeatureStatus().await()) {
FeatureStatus.AVAILABLE -> summarize()
FeatureStatus.DOWNLOADABLE -> summarizer.downloadFeature(callback)
else -> showFallback()
}
Key constraints: inference only while your app is in the foreground (BACKGROUND_USE_BLOCKED), a per-app quota (BUSY), no support on devices with an unlocked bootloader, and a summarization input limit of 4,000 tokens.
On-Device vs Cloud
| Factor | On-device | Cloud |
|---|---|---|
| Privacy | Data stays on the phone | Data leaves the device |
| Latency | No network round trip | Network dependent |
| Cost | No per-token fee | Pay per request |
| Offline | Works offline once downloaded | Needs a connection |
| Model quality | Small model, short context | Large models, long context |
| Battery | Uses device power and heat | Minimal device cost |
| Reach | Supported devices only | Every device |
Practical Integration Tips
- Detect, then decide. Never show an AI button before checking availability. Hide it or route to a fallback.
- Build a fallback ladder: on-device model, then a cloud model (with user consent), then a non-AI path.
- Write narrow prompts. One task per request, trusted rules in instructions, explicit length ("in one sentence"), structured output over free text.
- Handle downloads. The model may still be downloading on first launch; show progress or retry later.
- Test like a non-deterministic system. Keep a fixed set of sample inputs, re-run it on every OS or model update, and test on unsupported devices.
- Handle refusals. Safety guardrails and context overflow throw errors; catch them and show a helpful message.
Privacy Labels and Data Safety
Data processed only on the device is generally not "collected" for Apple's App Privacy details or Google Play's Data safety form. That changes as soon as prompts or outputs leave the device: cloud fallbacks, logging prompts to analytics, crash reports containing user text, or third-party model providers. Disclose those flows, describe AI features in your privacy policy, and keep user text out of logs.