Troubleshooting

Common issues and fixes. For desktop-specific problems (Linux native logs, glibc, Windows DXC, stale GPU shader cache) see Desktop Support → Troubleshooting.

Downloads#

• Resume isn't supported by the HuggingFace CDN. flutter_gemma uses smart retry with exponential backoff and automatic restart of interrupted downloads instead. Tune the attempt count via maxDownloadRetries in FlutterGemma.initialize(...) (default: 10).
• Large downloads on Android need foreground: true to get a foreground service (which shows a notification) and bypass Android's 9-minute background execution limit. The default (null) does not configure a notification, and without one the platform never starts the service — so the size threshold alone does nothing. iOS uses native URLSession and needs no special handling. See Models → downloads.
• Using background_downloader in your own app too? flutter_gemma no longer listens to FileDownloader().updates (fixed in flutter_gemma 1.6.3). That stream takes a single subscription, so claiming it made every later FileDownloader().updates.listen(...) in the host app throw "Stream has already been listened to". Updates are now scoped to flutter_gemma's own task group and the stream stays yours.
• Custom servers on Web must enable CORS headers. HuggingFace is already configured correctly; for Firebase Storage see the CORS configuration docs.

Gated models / download errors (401, 403)#

installModel(...).install() throws a public DownloadException carrying a sealed DownloadError, so you can react to gated HuggingFace models (HTTP 401/403) by type instead of substring-matching error strings:

try {
  await FlutterGemma.installModel(modelType: ModelType.gemmaIt)
      .fromNetwork(url, token: hfToken)
      .install();
} on DownloadException catch (e) {
  switch (e.error) {
    case UnauthorizedError():   // 401 — missing/invalid HuggingFace token
    case ForbiddenError():      // 403 — token lacks access to this gated model
      showGatedModelDialog();
    case NotFoundError():       // 404 — bad URL (use /resolve/main/, not /blob/main/)
      showNotFoundDialog();
    case RateLimitedError():    // 429
    case ServerError():         // 5xx
    case NetworkError():        // connectivity
    case CanceledError():       // user canceled
    case UnknownError():
      showRetryDialog(e.error.toUserMessage());
  }
}

For a gated model (401/403): pass a valid huggingFaceToken to FlutterGemma.initialize(...) (or token: on fromNetwork(...)), open the model page on HuggingFace, accept its license, and request access. Each DownloadError also exposes toUserMessage(), toTitle(), isRetryable, and requiresUserAction for building UI.

Memory#

• iOS: ensure Runner.entitlements contains the memory entitlements and the deployment target is at least 15.0 (16.0 with flutter_gemma_mediapipe) — in Xcode on the Runner target under SPM, or in the Podfile when one exists. See Installation → iOS.
• Reduce maxTokens if you hit memory pressure — but keep it at 1024 or higher for .litertlm models (see "maxTokens vs maxOutputTokens" below). To shorten replies, use maxOutputTokens, not a smaller maxTokens.
• Use smaller models (1B-2B parameters) for devices with <6GB RAM. Multimodal models (Gemma 4, Gemma3n) need 8GB+.
• Close sessions and models when not needed; monitor usage with sizeInTokens().

maxTokens vs maxOutputTokens#

maxTokens (on getActiveModel/createModel) is the context window — the total budget shared by the input (system prompt + history + your message) and the generated output (the KV-cache size). It is not the reply length.

.litertlm models require a context window of at least 1024. Passing a smaller maxTokens (e.g. 100) used to crash with DYNAMIC_UPDATE_SLICE failed to prepare / Failed to allocate tensors (even on CPU — the failure is at graph compile, before the backend runs). As of 1.0.2 a too-small maxTokens is clamped up to 1024 automatically with a log warning.

To limit how many tokens the model generates, use maxOutputTokens on createSession/openSession/createChat/openChat instead:

final model = await FlutterGemma.getActiveModel(maxTokens: 1024); // context window
final chat = await model.createChat(maxOutputTokens: 100);        // reply cap

(maxOutputTokens is honored on .litertlm; the MediaPipe .task path has no session-level output cap and ignores it.)

iOS#

• Build issues: ensure the minimum iOS version is at least 15.0 (16.0 with flutter_gemma_mediapipe). If the app has a Podfile (any app using flutter_gemma_mediapipe does — it ships no Package.swift), also use static linking (use_frameworks! :linkage => :static) and reinstall with cd ios && pod install --repo-update. An SPM-only app has no Podfile; there the deployment target lives on the Runner target in Xcode.
• Simulator GPU disabled: iOS Simulator's Metal has a 256 MB single-allocation cap that LLM weight tensors exceed (e.g. Gemma 3 1B's KV cache alone is 288 MB). Use CPU on the simulator, or test GPU on a physical iPhone. This is a simulator limit, not a plugin bug.

Android#

• .litertlm models require minSdk 30. libLiteRtLm.so depends on API 30+ Bionic syscalls (pthread_cond_clockwait, sem_clockwait) that can't be shimmed on older devices. MediaPipe .task models work on lower API levels.
• .litertlm / embeddings / vision are arm64-v8a only. MediaPipe text inference (.task / .bin) also runs on x86_64 and armeabi-v7a. If you only use arm64-only features, add ndk { abiFilters 'arm64-v8a' } so the Play Store doesn't offer broken APKs. See Installation → Android architecture.
• GPU: nothing to add — the OpenCL <uses-native-library> entries come from flutter_gemma's own manifest (1.2.0+) through the manifest merger. If the GPU backend still falls back, check that the merged manifest contains libvndksupport.so and libOpenCL.so. See Installation → Android.
• Zero chunks and Stream error: <U+FFFD>, then SIGABRT. Fixed in flutter_gemma_litertlm 1.5.2. On Android the first dlopen of libLiteRtLm decides for the whole process whether its symbols are reachable from the default search scope, and bionic never promotes it afterwards — so an app that embedded or transcribed anything before its first generation left the stream-callback ABI probe blind and the wrong callback shape was registered. Upgrade to 1.5.2. If your own or third-party code loads libLiteRtLm first, load it with RTLD_GLOBAL — 1.5.2 cannot repair that case, but it raises a StateError naming it rather than generating corrupt text. See #447.

Web#

• MediaPipe is GPU-only on web. The web engine ignores preferredBackend and always runs on the browser's GPU (WebGPU). ONNX on web is the exception: PreferredBackend.cpu pins WASM there.
• Mobile .task models often don't work on web — use the -web.task (MediaPipe) or .litertlm (LiteRT-LM) web variant.
• Memory / cache limits:

• Large models (>2GB): use WebStorageMode.streaming (OPFS) to bypass the ~2 GB blob limit. Check support with await FlutterGemma.isStreamingSupported(). See Installation → web storage.
• Storage modes: cacheApi (default, persists across restarts, <2GB), streaming (OPFS, large models, requires Chrome 86+/Edge 86+/Safari 15.2+), none (ephemeral, testing only).

Web .litertlm (early preview) feature matrix#

Web .litertlm inference runs Gemma .litertlm models in the browser via the upstream @litert-lm/core package (WebGPU + WASM). It is an early preview and a subset of the native path. MediaPipe .task on web is unaffected and remains fully supported.

Works on web .litertlm: text generation (sync + streaming), multi-turn chat with history, system instruction, function calling / tool calls (Gemma 4), concurrent sessions (serialized), large models via OPFS streaming, GPU only.

Not supported on web .litertlm yet (mobile/desktop only):

• ❌ Vision / image input — image inputs are dropped with a debug warning.
• ❌ Audio input — no Audio executor config in the JS API.
• ⚠️ Thinking mode — Qwen3 and DeepSeek R1 reasoning is still split out of the token stream (that part is pure Dart). Gemma 4 thinking needs the native extraContext channel: the web engine passes it to @litert-lm/core, but it has never been verified end to end — treat it as unsupported until it is.
• ❌ LoRA weights — loraPath throws UnsupportedError.

Windows desktop GPU crashes#

Windows embeddings and speech fail with status 3#

NPU#

The model answers, fluently, about the beginning of a long prompt and ignores the rest. That is the Gemma 3 family on an NPU. Nothing raises, nothing is logged: every prefill chunk after the first is dropped, so the model genuinely never saw the end of your prompt. On Qualcomm it is a size threshold in the compiled bundle's prefill mask (~1 MiB; at prefill 128 the largest working cache_length for the 4-head Gemma 3 bundles is 896, and every published qualcomm.* Gemma 3 bundle is built above it); on Intel it happens regardless of size, through a different defect. Run a Gemma 4 bundle on the NPU, or move that model to PreferredBackend.cpu / .gpu. Upstream: LiteRT-LM#3508.

Note that maxTokens is not clamped up to 1024 on the NPU attempt the way it is on CPU and GPU — the safe context is baked into the compiled bundle, so pass the cache_length it was built for. If the NPU fails to initialize the engine falls back to GPU and then CPU, and the floor applies to those attempts, so the fallback is clamped rather than crashed.

PreferredBackend.npu is unavailable on a recent Snapdragon. SoC coverage is the runtime's, not ours: Snapdragon 8 Gen 5 (SM8845 — OnePlus 15R, iQOO 15R and the like) is not covered upstream, and a context compiled for SM8850 is rejected by an SM8845 device even though both are Hexagon v81. Tracked at LiteRT#7516; note the easily-confused naming — 8s Gen 4 is SM8735, not SM8845.

Desktop storage locations#

Desktop builds store downloaded models outside the user's Documents/ folder to avoid OneDrive / iCloud / Domain-Roaming sync corrupting FFI mmap of large .litertlm files:

• Windows: %LOCALAPPDATA%\flutter_gemma\ (never OneDrive-synced)
• macOS: ~/Library/Application Support/<bundle>/flutter_gemma/
• Linux: ~/.local/share/<app>/flutter_gemma/

Models installed by older 0.14.x / 0.15.0 builds that still live under Documents/ keep working via a fallback read.

On Windows, flutter_gemma before 1.4.0 could write a fresh download to a $CWD-relative path (<cwd>\Users\…\AppData\Local\flutter_gemma\) instead of the absolute %LOCALAPPDATA%\flutter_gemma\, because %LOCALAPPDATA% is not one of background_downloader's base directories. The model then reported "installed" but failed to load with "model file paths not found". Fixed in 1.4.0 — it affected every fresh inference / embedding / STT download on Windows, so upgrade if you hit it.

Multimodal#

• Ensure you're using a multimodal model (Gemma 4, Gemma3n E2B/E4B, FastVLM).
• Set supportImage: true (and supportAudio: true for audio) when creating the model.
• Check device memory — multimodal models require more RAM.
• Image input crashes at model load on a GPU text backend (older releases). On Metal (iOS/macOS) and WebGPU (Windows/Linux) the vision encoder's ops can't be prepared by the GPU delegate. Fixed in flutter_gemma_litertlm 1.4.2 / core 1.5.9 — the vision encoder now defaults to CPU while the text decoder keeps the GPU. Upgrade if you hit it.
• Use the GPU backend for faster text decoding. Image encoding runs on CPU by default; move audio encoding to GPU with preferredAudioBackend: PreferredBackend.gpu. To force GPU vision (only for a model built to allow it), pass preferredVisionBackend: PreferredBackend.gpu. See Multimodal.

Native libraries fetched at build time#

Some packages download their native library from a GitHub Release when you first build for a platform, then cache it under ~/.cache/flutter_gemma/native/ (~/Library/Caches/… on macOS, %LOCALAPPDATA%\… on Windows). This applies to flutter_gemma_litertlm (always has), flutter_gemma_onnx, and flutter_gemma_rag_sqlite from 1.3.0 — before that it shipped the loadables inside the package.

• The build fails with a download error or an HTTP status. The first build of each platform needs github.com reachable. In an air-gapped or proxied CI, copy the whole flutter_gemma/native cache directory from a machine that built the same package versions, including its hidden version-marker files — a folder copied without them is discarded and fetched again. There is no setting that points the build at archives you vendor yourself.
• The build fails with CHECKSUM MISMATCH. The bytes served do not match what the package version was pinned to. Re-run once to rule out a corrupt transfer. If it persists, the release asset was replaced after publication — do not work around it by clearing the checksum; report it.
• A build that used to succeed now fails instead of quietly skipping. That is deliberate. These hooks used to report success while bundling nothing, which surfaced later as an opaque dlopen crash on a user's device. A platform the package claims to support now fails the build when its library cannot be produced.
• Maintainers only: a local native/<name>/prebuilt/<target>/ overrides the pinned release. flutter_gemma_rag_sqlite says so on stderr when it takes that path; flutter_gemma_litertlm and flutter_gemma_onnx take it silently. If a new release "did not take", look for that directory first.

Function calling#

• Function calling is supported only by select models (Gemma 4, Gemma3n, Gemma 3 1B, FunctionGemma, DeepSeek, Qwen, Phi-4). Unsupported models log a warning and ignore tools — they still work for text generation. Check supportsFunctionCalls. See Function Calling.