Cookbook

Copy-pasteable Kotlin for each thing the SDK does. Every snippet uses only the public facade — MobileTransformers, MobileTransformerModel, and the config/ types. No ORT*, *Native or *Repository type appears here, and none should appear in your app either.

Each recipe mirrors a screen in the sample app (MobileTransformersApp), so the documentation and the worked example cannot describe different APIs. The screen is named under each heading.

Device reality. arm64-v8a only; there is no x86_64 emulator build. Most calls below do nothing until a package is installed, so start with the first recipe.

---

1. Pull a model from the Hub, with progress

Sample app: Models screen.

fromPretrained installs the package when it is missing, then loads it. Download progress is reported through DownloadProgress; fraction is null until the download plan is known, which is the honest state for "total not yet resolved".

val model = MobileTransformers.fromPretrained(
    context = context,
    repoId = "HuggingFaceTB/SmolLM2-135M-Instruct",
    features = setOf(ModelFeature.Inference),
    onDownloadProgress = { progress ->
        Log.i("pull", "${progress.filesDone}/${progress.filesTotal} ${progress.path}")
    },
)

Ask for a feature only if you need it. Requesting one the package does not ship fails closed with FeatureNotInstalledException at construction rather than at first use.

Private or gated repos

Pass a HubConfig. Without one the pull is anonymous, and a private repo fails with a 401 that looks like any other network error:

val model = MobileTransformers.fromPretrained(
    context = context,
    repoId = "your-org/your-private-package",
    hubConfig = HubConfig(token = yourToken),
)

Where yourToken comes from is your app's decision, and it matters. A production app should obtain one at runtime — from the user, or from a backend that authenticates them — and never store it in the APK. The sample app takes the development shortcut instead, and says so: its build.gradle.kts reads the HF_TOKEN environment variable at build time into BuildConfig.HF_TOKEN.

HF_TOKEN=hf_xxx ./gradlew :MobileTransformersApp:assembleDebug
# or, without exporting it:
./gradlew :MobileTransformersApp:assembleDebug -PmtHubToken=hf_xxx

A token compiled into an APK is extractable — strings over the dex is enough. That is acceptable for reaching your own private repo on your own device, and not acceptable for anything you distribute.

What is already installed?

MobileTransformers.installed(context).forEach { pkg ->
    Log.i("cache", "${pkg.sanitizedRepoId} ${pkg.sizeBytes / (1024 * 1024)} MB ${pkg.variantIds}")
}

---

2. Generate, with streaming

Sample app: Chat screen.

val result = model.generate(
    prompt = "The capital of France is",
    config = GenerationConfig(
        maxNewTokens = 64,
        sampling = SamplingConfig(method = SamplingMethod.GREEDY),
    ),
    callback = object : GenerateCallback {
        override fun onPartialResult(progress: GenerateProgress) {
            print(progress.token)          // token-by-token, in order
        }
    },
)
Log.i("gen", "${result.tokenCount} tokens at ${result.avgTokensPerSecond} tok/s")

---

3. Choose an engine

Sample app: Chat screen, engine picker.

Native is the guaranteed floor. GenAI is selectable only when all three hold: the installed package ships inference/genai_config.json, its manifest variant declares genai in supportedEngines, and the device's GenAI probe succeeds. Ask, rather than guessing:

if (InferenceEngine.GENAI in model.capabilities.availableEngines) {
    // reload with the other engine; the engine is fixed at load time
    val genai = MobileTransformers.fromPretrained(
        context = context,
        repoId = repoId,
        engine = InferenceEngine.GENAI,
    )
}

Naming an engine you cannot have raises EngineUnavailableException instead of quietly giving you Native. Silent substitution is what made an earlier engine-parity test compare Native with Native and pass.

Most packages are Native-only, including ones that ship a genai_config.json. Gemma-3 packages (FunctionGemma among them) are exported through optimum's main_export rather than the vendored GenAI builder, so their manifests declare supportedEngines: ["native"] even though optimum writes a genai_config.json beside the graph. availableEngines applies the manifest condition too, so it and the loader always agree — check it and offer only what it contains.

---

4. Fine-tune on device, then merge

Sample app: Train screen.

Use trainingJob() when you want status, events, cancellation or resume; train() is the one-shot convenience.

val job = model.trainingJob()

launch { job.status.collect { status -> Log.i("train", status.toString()) } }

job.start(
    dataset = DatasetConfig(trainFile = "my_data", task = "mobile_actions", maxSequenceLength = 160),
    config = TrainConfig(
        maxSteps = 120,
        batchSize = 2,
        learningRate = 5e-4f,
        // The optimizer steps on `globalStep % gradientAccumulationSteps == 0`. At the default of 4
        // a short bounded run can finish, report success, and apply no update at all.
        gradientAccumulationSteps = 1,
        mergeAtEnd = true,
    ),
)

Two things worth knowing before you size a run:

• maxSteps is an upper bound. Training also stops at the end of the epoch, so rows / batchSize wins when it is smaller — a run asking for 120 steps over 108 rows at batch 2 takes 54.
• Cancelling is resumable. job.cancel(saveCheckpoint = true) sets a cooperative flag; the loop breaks at the next step boundary and writes a checkpoint, and job.canResume then reads true.

Memory: training defaults to low_mem, and you should leave it there

TrainConfig().device.memoryConfigId is MemoryConfigId.LOW_MEM, unlike GenerationConfig's HIGH_PERF. That is not a conservative guess — HIGH_PERF enables ORT's memory-pattern planner and CPU arena, which on a training session pre-allocates the whole backward activation plan and holds its peak for the life of the run. Measured on a 5.5 GB phone, FunctionGemma-270M (~1.07 GB of fp32 weights, a 368,640-parameter LoRA) reached 2.35 GB RSS + 1.02 GB swap under HIGH_PERF and was killed by the system; under LOW_MEM the same run completes.

There is no exception to catch when that happens: Android sends SIGKILL, so the process vanishes with no error, no finally and no checkpoint. If you override this, do it knowing that is the failure mode:

// Only if you have measured that it fits.
TrainConfig(device = DeviceConfig(memoryConfigId = MemoryConfigId.HIGH_PERF))

Inference keeps HIGH_PERF: a forward-only session benefits from the arena and builds no backward plan. If you fan one DeviceConfig across every config in your app, exclude the training memory profile — the sample app's AppConfig.updateDevice shows the shape.

Train while charging

TrainingScheduler.schedule(
    context = context,
    repoId = model.repoId,
    dataset = DatasetConfig(trainFile = "my_data", task = "cola"),
    training = TrainConfig(maxSteps = 500),
    config = TrainingScheduleConfig(
        // "Not before", NOT an appointment — see below.
        initialDelayMinutes = 240,
    ),
)

Each chunk re-enters the WorkManager queue, so unplugging pauses the run rather than failing it.

On start times. initialDelayMinutes maps to WorkManager's setInitialDelay, which is the only start-time control Android gives deferrable work, and it is a floor: the system batches, and Doze can hold a job well past it. An exact wall-clock start would need AlarmManager.setExactAndAllowWhileIdle and the SCHEDULE_EXACT_ALARM permission, which Play restricts to alarm clocks and calendar reminders — a background trainer is neither. The constraints (requiresCharging, requiresBatteryNotLow) are the real gate; the delay only moves the earliest moment they are consulted. It applies to the first chunk only, so a multi-chunk run is not re-delayed at every boundary.

---

5. Ground answers in your own documents

Sample app: Chat screen, RAG toggle.

model.ingest(path = "/sdcard/Download/notes.md", config = RagConfig())

val grounded = model.generateWithRag(
    query = "what did I write about batching?",
    rag = RagConfig(topK = 5, minScore = 0.2),
    generation = GenerationConfig(maxNewTokens = 200),
    promptStrategy = PromptAssembler.DEFAULT,
    // Optional, and worth passing in any UI: a grounded turn does an embedding pass, a vector search
    // and then a long decode, so without this the screen shows nothing until all three are done.
    callback = object : GenerateCallback {
        override fun onPartialResult(progress: GenerateProgress) = append(progress.token)
    },
)

Log.i("rag", grounded.text)
// `title` is the ingested file the passage came from; several matches can share one.
grounded.matches.forEach { Log.i("rag", "${it.score} ${it.title} ${it.text}") }
Log.i("rag", "asked: ${grounded.prompt}")   // the assembled prompt, for when the answer is wrong

Leave the embedding identity unset unless you mean to override the package: it is read from embedding/rag_config.json, written by the exporter from the encoder it actually shipped.

---

6. Tool calls: instruction → validated call → dry-run intent

Sample app: Tool calls screen.

Your app declares the actions. A model selects an action; it cannot name an intent — the intent string comes from your ActionSpec — so the reachable set of intents is fixed when you write this list.

val validator = FunctionCallValidator(
    listOf(
        ActionSpec(
            actionName = "set_alarm",
            parameters = mapOf("time" to "string"),
            allowedIntent = "android.intent.action.SET_ALARM",
            validationRules = mapOf("time" to "HH:mm"),
        ),
    ),
)

when (val result = model.generateToolCall("wake me at 07:30", validator)) {
    is ToolCallResult.Accepted -> {
        val intended = result.dryRun()
        Log.i("tool", "${intended.intent.action} willExecute=${intended.willExecute}")  // false
    }
    is ToolCallResult.Rejected -> Log.i("tool", "refused: ${result.reason}")
    is ToolCallResult.NoCall -> Log.i("tool", "answered in prose: ${result.raw}")
}

Three outcomes, and the third is not a refusal. NoCall means the model answered in words rather than attempting a call — nothing was permitted or denied. Reporting that as Rejected (which this API used to do, with reason = "no tool call found in the model's output") tells a user their allowlist blocked something when it did not, and it hides format mismatches: a parser reading the wrong dialect produces NoCall for every well-formed call.

That distinction is what makes this usable as your only chat path: declare the tools on every turn and let the outcome decide how the turn renders, instead of asking the user to predict, before sending, whether their message is a tool call.

Rejected is a value, not an exception: refusing untrusted output is the expected path. Nothing here executes — IntentBinder holds no Context and has no startActivity call site, so firing the intent is your deliberate act with your own Context.

Dialects: check what the model speaks

Not every model emits JSON. FunctionGemma emits <start_function_call>call:name{key:<escape>value<escape>}<end_function_call>, and handing that to a JSON reader yields NoCall for calls that are perfectly well formed. The dialect is detected from the package's own chat template:

if (model.capabilities.supportsToolCalling) {
    Log.i("tool", "grammar: ${model.capabilities.toolCalling.dialect}")  // FUNCTION_GEMMA | JSON
}

generateToolCall defaults its parser from that, and ToolPromptBuilder renders the declarations — and the turn structure — in the matching grammar. Pass parser = only if you know better than the package does.

supportsToolCalling being false does not forbid tool calls: a model fine-tuned on this repo's mobile_actions corpus learns the JSON shape without its template ever mentioning tools. It means only that the model has no grammar of its own, so an app should not advertise the capability.

Build the training set from the same declaration so the corpus and the boundary are provably one value:

mobiletransformers agent-dataset --source generated \
  --allowlist build/agent/action_schema.json --output build/user

Status, 2026-08-15. The on-device gate for this recipe (ToolCallDeviceTest) passes — 2 tests / 0 failures / 754 s on an S21 FE, steps=108 lossDrop=99.5%. The repeated-newline failure this note used to describe was the merge-transpose defect, fixed 2026-08-14: the model had learned the task all along and the merge was corrupting the result on the way out. FunctionGemma has since been observed emitting a well-formed call in its own grammar on the same device.

---

7. One federated round

Sample app: Federated screen.

Federation is off by default (BuildConfig.FEDERATION_ENABLED). The round returns bytes and accepts bytes; the transport is yours, which is what lets the whole loop run against a local federated serve.

val result = model.federatedRound(
    config = FederatedConfig(
        gatewayUrl = "https://gateway.example",
        clientAuthToken = token,
        consent = FederatedConsent(
            granted = true,
            policyVersion = "1.0",
            grantedAtEpochMs = System.currentTimeMillis(),
        ),
    ),
    globalRecord = previousAggregate,   // null for round 0
    roundNumber = 1,
    localTraining = { round -> model.train(dataset, TrainConfig(maxSteps = 20)) },
)

upload(result.update)                    // result.payloadBytes is what federation costs per round

Consent, TLS and auth are checked before any tensor is read, and the refusal names the missing protection. Only adapter factors and aggregate metrics ever leave the device — never your examples.

---

8. Close it

model.close()

A model owns native sessions. Load once and share the handle; loading the same package twice opens two sessions over one set of weights.