I'm building a life OS that runs a 4B parameter AI model on your phone — zero cloud, zero subscription, zero data leaving your device

Your phone is already powerful enough to run a 4-billion-parameter language model. Most app developers haven't noticed yet.

I'm building LifeOS — a personal life operating system where every AI feature runs entirely on-device. Smart notifications generated by a local LLM. Email-to-task extraction that never sends your Gmail to a server. A weekly review written about your specific habits, by a model that lives in your pocket, costs $0 per inference, and can't be rate-limited, deprecated, or paywalled.

The constraint I set before writing a single line: no cloud AI. Ever. Not as a fallback. Not as a degraded path. Not at all. If Gemma can't do it on the device, the feature doesn't ship.

That rule made this project significantly harder, and I think that's exactly why it's worth building.

What LifeOS actually is

Most productivity apps are glorified reminders with a GPT button bolted on. The notification says "Gym at 6pm." The AI feature, if there is one, calls an API and returns a slightly longer string.

LifeOS is built around a different premise: the AI is the architecture, not a feature. Every intelligent decision the app makes — which notification is worth interrupting you for, how to time-block your afternoon given your energy curve, whether a deadline is actually reachable given your historical pace — is made by an LLM running in real time on your device.

The app connects to Google Calendar and Gmail, manages tasks and habits, schedules context-aware alarms, and uses Gemma 4's language understanding to make decisions that feel genuinely personal. Not "Gym reminder" — "You haven't hit the gym since Monday. A 30-minute session today breaks the streak gap and your calendar shows a free 5 PM slot."

That kind of output isn't possible with static notification logic. It requires a model that can read context, reason about patterns, and generate language. It's happening on your A-series chip or Snapdragon NPU, in under two seconds, with no round-trip.

The hardest technical problem: running a 4B model on a phone

Let's start with the centerpiece, because everything else in the architecture flows from it.

The model is Google Gemma 4 Edge 4 Billion (Gemma 4 E4B) — Apache 2.0 licensed, 4B parameters, quantized to Q4 (approximately 2.5 GB). The quantization is done via GGUF format, which reduces the weight precision from 16-bit floats to 4-bit integers. You lose some quality at the tail end of complex reasoning. For the structured JSON generation this app needs, the quality loss is barely measurable.

Not bundled in the binary. The App Store has a 4 GB install size limit, and a 2.5 GB model in the binary would make every update redownload 2.5 GB. The model downloads on first launch with a progress screen. Once it's on-device, it never leaves.

The inference runtime is different on each platform, and building both bridges is where this gets genuinely hard.

Android: LiteRT-LM with a Kotlin native module

On Android, inference runs through LiteRT-LM — Google's rebranded TensorFlow Lite engine, specifically optimized for LLMs. It has hardware acceleration for Qualcomm Snapdragon and MediaTek Dimensity NPUs, which matters because mobile AI inference without hardware acceleration is battery-draining and slow.

The bridge is a React Native native module written in Kotlin:

// android/app/src/main/java/com/lifeos/GemmaModule.kt
class GemmaModule(reactContext: ReactApplicationContext) :
    ReactContextBaseJavaModule(reactContext) {
 
    private var session: LlmInference? = null
 
    @ReactMethod
    fun generate(prompt: String, systemPrompt: String?, promise: Promise) {
        coroutineScope.launch {
            try {
                val options = LlmInference.LlmInferenceOptions.builder()
                    .setModelPath(modelPath)
                    .setMaxTokens(512)
                    .setTemperature(0.7f)
                    .build()
 
                val result = LlmInference.createFromOptions(context, options)
                    .generateResponse(buildFullPrompt(systemPrompt, prompt))
 
                promise.resolve(result)
            } catch (e: Exception) {
                promise.reject("GEMMA_ERROR", e.message, e)
            }
        }
    }
}

Writing this module from scratch means owning the full call chain: React Native JS → JSI bridge → Kotlin → LiteRT-LM SDK → GPU/NPU. Most React Native developers use third-party packages so they never see this layer. I needed custom control over session management, model path resolution, and error propagation — so I wrote it directly.

iOS: Core ML + the Neural Engine

On iOS, the inference path is different. Apple's Neural Engine (ANE) — the dedicated ML accelerator on every A-series and M-series chip — is the most efficient inference hardware in the mobile space. An iPhone 15 Pro runs Gemma 4 E4B at ~10 tokens/second on the ANE with minimal battery impact. That's fast enough for every use case in this app.

The catch: Core ML doesn't natively consume GGUF weights. You have to convert:

# Conversion pipeline on Mac using Apple's coremltools
import coremltools as ct
import torch
 
# Load the GGUF-quantized weights
model = load_gemma_gguf("gemma-4-e4b-q4.gguf")
 
# Trace and convert to Core ML format
traced = torch.jit.trace(model, example_inputs)
mlmodel = ct.convert(
    traced,
    compute_precision=ct.precision.FLOAT16,
    compute_units=ct.ComputeUnit.ALL,  # Use ANE + GPU + CPU
)
mlmodel.save("Gemma4E4B.mlpackage")

The Swift native module wraps this:

// ios/LifeOS/GemmaModule.swift
@objc(GemmaModule)
class GemmaModule: NSObject {
    private var model: MLModel?
 
    @objc func generate(
        _ prompt: String,
        systemPrompt: String?,
        resolve: @escaping RCTPromiseResolveBlock,
        reject: @escaping RCTPromiseRejectBlock
    ) {
        Task {
            guard let model = self.model else {
                reject("MODEL_NOT_READY", "Gemma model not loaded", nil)
                return
            }
            // Core ML prediction through ANE
            let result = try await runInference(model: model, prompt: buildPrompt(systemPrompt, prompt))
            resolve(result)
        }
    }
}

The JS interface that unifies both platforms:

// src/services/ai/GemmaService.ts
interface GemmaService {
  isReady(): Promise<boolean>
  generate(prompt: string, systemPrompt?: string): Promise<string>
  generateJSON<T>(prompt: string, schema: object): Promise<T>
}

generateJSON is the real workhorse. Every AI feature in the app produces structured output — task arrays, schedule blocks, notification scores — through this single method. Unstructured generation is only used for the weekly review copy.

The stack, with actual reasons

React Native + Expo SDK 52. Expo's managed workflow means I'm not writing Podfile patches and Gradle config by hand for the non-AI parts of the app. The exception is the Gemma native modules — those aren't in the Expo ecosystem, so I ejected to bare workflow for those files specifically. This is the correct level of ejection: take native control only where you need it, keep Expo handling everything else.

TypeScript strict mode, no exceptions. No any without an explanation comment. No type assertions without a comment. This is non-negotiable on a codebase with this many boundaries — JS ↔ Kotlin, JS ↔ Swift, AI output ↔ app state. Every interface between layers is a place where untyped data can corrupt the database or crash the UI.

WatermelonDB over SQLite directly. WatermelonDB is a reactive, offline-first SQLite wrapper that was built specifically for React Native. The key behaviors I needed:

• Lazy loading: only loads records currently rendered. A task list with 2,000 items doesn't hit SQLite for all 2,000 rows.
• Reactive queries: components re-render when underlying data changes, without manual subscription management.
• Processed ID tracking: every email processed by the Gmail extraction feature gets its ID stored. Gemma never sees the same email twice.

AsyncStorage is explicitly banned for structured data. It's not queryable, not reactive, and doesn't scale past a few hundred kilobytes without degrading.

Zustand v5 over Redux. A Zustand store is a function. There's no action creator, no reducer, no dispatch pattern to learn. Four stores: tasksStore, habitsStore, userStore, aiStore. The AI store tracks model loading state, inference queue depth, and the last known model error. No Redux.

Zod v3 for every AI output. This is the defensive engineering decision I'm most proud of. Every JSON object Gemma generates gets validated through a Zod schema before it touches the database:

const TaskSuggestionSchema = z.object({
  title: z.string().min(1).max(200),
  category: z.enum(TASK_CATEGORIES),
  priority: z.enum(['high', 'medium', 'low']),
  dueAt: z.string().datetime().optional(),
  estimatedMinutes: z.number().int().positive().optional(),
})
 
// If Gemma hallucinates a field, this throws before the DB write
const parsed = TaskSuggestionSchema.parse(rawGemmaOutput)

Language models hallucinate. Quantized 4B parameter models hallucinate more. Zod is the firewall between model output and database integrity.

Firebase v11 for sync only. Firebase handles cross-device habit and task sync — but it is not the AI layer and never will be. Local WatermelonDB is the source of truth. Firebase is a sync layer. If Firebase is unavailable, the app works fully — you just don't get cross-device sync. That hierarchy matters for an offline-first product.

NativeWind v4 (Tailwind for RN). StyleSheet.create() is banned. Tailwind classes throughout. The design system is consistent and responsive styling is readable.

What changes when you remove cloud AI

This is the question I get asked most: what's the actual engineering difference between calling Gemma locally versus calling Claude via API?

The answer is: everything about how you write prompts and design features.

Token budget is real. A cloud API call can accept 100,000 tokens of context without blinking. On-device inference with a 4B model has hard context window limits, and longer prompts mean slower inference. Every prompt in this app is engineered to be short and information-dense. For the daily planner feature, the prompt is:

Today's date: {date}. Energy curve: {peak_hours}. 
Calendar events (JSON): {events}
Pending tasks with estimates (JSON): {tasks}
Output a time-blocked schedule filling free slots. 
Return JSON array matching: {schema}

No preamble. No "You are a helpful assistant." No lengthy role description. Gemma 4 E4B is instruction-tuned — it understands the task from the schema alone.

Structured output is non-negotiable. With cloud APIs you can tolerate some freeform output and parse it downstream. With on-device inference at low token counts, you want a JSON object on the first try. All prompts end with an explicit schema and "Return only valid JSON." generateJSON enforces this at the service layer.

Graceful degradation, not silent fallback. If the model fails to load (device RAM too low, corrupted download), the AI features disappear. They don't silently call a cloud API. This was a deliberate product decision — the privacy guarantee has to be absolute. A user who chose LifeOS for privacy reasons should never discover their data was sent to a cloud server because their phone ran out of RAM.

The architecture decisions nobody talks about

Prompt templates as named exports. All Gemma prompts live in src/services/ai/prompts/ as named TypeScript constants. No prompt strings scattered through components. No prompts assembled on the fly from string interpolation. Every prompt is reviewable, diffable, and testable in isolation.

Hard rate limit on notifications: 3 per hour. The SmartNotifService.ts enforces a hard cap regardless of what the AI scoring returns. I've been burned by notification fatigue in every other productivity app I've used. The cap is not configurable by the user — it's an architectural constraint. A coach who texts you 20 times a day isn't a coach, they're a problem.

Email extraction is always human-approved. The F6 (Gmail extraction) feature never auto-creates tasks. It surfaces an "Inbox Review" screen where every AI suggestion gets an approve/dismiss card. The rationale: a feature that silently adds tasks to your list based on email content is a liability. One bad extraction and users lose trust in the entire product. The approval gate is the trust layer.

Actual minutes tracked for velocity. Every task has both estimatedMinutes (set at creation) and actualMinutes (computed from start/complete timestamps). The deadline prediction feature (F13) uses this to calculate a personal velocity multiplier — "you typically take 1.4x your estimate." This is behaviorally adaptive without any cloud data. All the modeling is local arithmetic over your own history.

What's actually hard (and I won't pretend otherwise)

Android fragmentation on the inference path. Qualcomm Snapdragon has hardware acceleration support in LiteRT-LM. MediaTek Dimensity has partial support. Google Tensor (Pixel phones) has a different acceleration path entirely. Older or low-end chips fall back to CPU inference, which is too slow to be practical. The honest answer is: LifeOS will have a minimum device spec, and I'm still benchmarking to find where the cutoff is.

Q4 quantization quality on complex reasoning. For "generate a task from this email," Q4 is fine. For "analyze my week and identify patterns in what I skipped," the quality degradation is more noticeable. I'm currently testing whether Q6 quantization (roughly 3.5 GB) improves the weekly review output enough to justify the larger model size. This is an active trade-off, not a solved problem.

iOS TestFlight with a 2.5 GB model download. TestFlight has its own download and installation behavior that interacts unexpectedly with large on-demand resources. Getting the model download flow right for iOS App Store review — where reviewers use test devices, not real ones — is going to be a distribution engineering problem that has nothing to do with AI.

Where it stands and what's next

Phase 1 (foundation + core features) is underway: project setup, Firebase auth, WatermelonDB schema, Zustand stores, Google OAuth. The Google Calendar sync service and task manager are the next two items.

Phase 2 (Gemma native modules + model download flow) comes after Phase 1 is stable and tested. The rule I've set: no AI feature gets built on a broken foundation.

Phase 3 (F6–F15 AI features) is where the interesting problems live. Email-to-task extraction with Gemma. The hourly smart notification scoring. Energy-based schedule generation. A weekly review that knows your specific patterns. All local, all free after download, all running on hardware you already own.

The app isn't in the store yet — I'm 2–3 months out, still researching the LiteRT-LM depth on Android and the Core ML conversion pipeline edge cases. I'm building this publicly because the design decisions are genuinely interesting, and because I want to document what it actually takes to build a production AI feature on the device rather than in the cloud.

If you're building in a similar space or thinking about on-device ML for your own app, I'd genuinely like to talk. The knowledge-sharing in this area is sparse compared to the cloud AI ecosystem.

Posts in this series will cover the Gemma native module implementation in detail, the prompt engineering for each feature, and the WatermelonDB schema design. Follow @bravim_builds if you want to see it built in real time.

Stack: React Native + Expo SDK 52 · TypeScript strict · Gemma 4 E4B (Q4, ~2.5 GB) · LiteRT-LM (Android/Kotlin) · Core ML + ANE (iOS/Swift) · WatermelonDB · Zustand v5 · Firebase v11 · NativeWind v4 · Zod v3 · expo-notifications