A voice assistant that runs entirely on the device. You ask it a question out loud; it transcribes you, thinks, and answers out loud — with the network switched off.
By the end you will have an app that:
Three models are in play, and most of what makes a voice app work well happens between them:
dart:ffi, which the browser does not have, so there is no web target in this codelabWatch out: This codelab starts from the finished app of Getting Started with On-Device LLMs — byte for byte, a CI check enforces it. If you have not done that codelab, step_01_starter still runs on its own; you will just be meeting the download-and-chat code for the first time.
Every step of this codelab exists as a complete, runnable app, so you can join at any point or check your work against the next one.
git clone --depth 1 https://github.com/DenisovAV/flutter_gemma.git
cd flutter_gemma/codelabs/voice-assistant-flutter-gemma
ls
step_01_starter/ the chat app you start from
step_02_hear/ after Step 2 — it listens
step_03_speak/ after Step 3 — it answers out loud
step_04_loop/ after Step 4 — one voice turn, end to end
step_05_barge_in/ after Step 5 — streamed speech you can interrupt
step_06_tools/ after Step 6 — it calls your functions
complete/ the finished app
Open step_01_starter and run it.
cd step_01_starter
flutter run
It is the finished app from Getting Started: it downloads a model once, then chats with it offline, streaming the reply as it is generated.
A voice assistant is that chat with two more models wrapped around it:
Steps 2 and 3 add the two new models. Step 4 hands all three to one object that runs them in order, and Steps 5 and 6 make the result feel like an assistant rather than a pipeline.
The starter defaults to Gemma 3 1B, which needs a Hugging Face token. For a voice assistant, use Gemma 4 E2B instead — Models.gemma4, already in model.dart as Getting Started's no-account option. It is ungated, and — this matters in Step 6 — it is the smallest Gemma that calls a tool and then says something sensible about the result.
Point the app at it in lib/main.dart:
const _model = Models.gemma4;
The web branch goes away with this line. Speech runs through dart:ffi, which the browser does not have, so from here on the app is Android, iOS and macOS.
flutter pub add flutter_gemma_speech record
flutter_gemma_speech runs the speech models; the API it implements — installStt, getActiveStt — is in core, the same split as the engine. record is the microphone.
await FlutterGemma.initialize(
inferenceEngines: [LiteRtLmEngine()],
sttBackends: [const LiteRtSttBackend()],
// huggingFaceToken and webStorageMode, as in the starter
);
Speech is opt-in the way engines are. Leave sttBackends out and getActiveStt() throws, naming the package to add.
lib/model.dart gets one more entry — moonshine tiny, English, 109 MB:
abstract final class Moonshine {
static const modelUrl =
'https://huggingface.co/litert-community/moonshine-tiny/resolve/main/'
'moonshine_tiny_5s_f32.tflite';
static const tokenizerUrl =
'https://huggingface.co/UsefulSensors/moonshine/resolve/main/'
'ctranslate2/tiny/tokenizer.json';
static const fileName = 'moonshine_tiny_5s_f32.tflite';
static const maxRecording = Duration(seconds: 5);
}
Two files, from two different repositories: the .tflite turns audio into token ids, and tokenizer.json turns those ids back into words.
Good to know: The 5s in the file name is the model's input window. Audio past five seconds is not transcribed — it is cut off. That is why the app stops recording at maxRecording rather than letting you ramble into a limit you cannot see.
DownloadPage installs the language model, then the recognizer:
await FlutterGemma.installStt()
.modelFromNetwork(Moonshine.modelUrl)
.tokenizerFromNetwork(Moonshine.tokenizerUrl)
.ofType(SttModelType.moonshine)
.withModelProgress((percent) {
if (mounted) setState(() => _percent = percent);
})
.install();
ofType picks the pipeline that runs the two files: moonshine, Whisper and Parakeet are three different ones. And the gate in main.dart now asks for both files before it lets the chat open:
Future<bool> _check() async =>
await FlutterGemma.isModelInstalled(widget.model.fileName) &&
await FlutterGemma.isModelInstalled(Moonshine.fileName);
This is the line that looks like a mistake. In ChatPage._load, after the chat opens:
await FlutterGemma.installStt()
.modelFromNetwork(Moonshine.modelUrl)
.tokenizerFromNetwork(Moonshine.tokenizerUrl)
.ofType(SttModelType.moonshine)
.install();
final stt = await FlutterGemma.getActiveStt();
The gate already made sure the files are here, so this downloads nothing. What it does is make moonshine the active speech model again. That choice lives in memory and does not survive a restart, and getActiveStt() throws without it. install() is idempotent: on files that are already there, it only re-activates them.
The recognizer takes one format: 16 kHz, one channel, 16-bit little-endian samples, no header. Ask the recorder for exactly that, as a stream:
final stream = await _recorder.startStream(
const RecordConfig(
encoder: AudioEncoder.pcm16bits,
sampleRate: 16000,
numChannels: 1,
),
);
final pcm = BytesBuilder(copy: false);
final done = Completer<void>();
stream.listen(pcm.add, onDone: done.complete, onError: done.completeError);
No file, no WAV to parse, no resampling — the bytes that arrive are the bytes transcribe wants. When you tap the mic again:
await _recorder.stop();
await _micDone?.future;
final text = (await _stt!.transcribe(_pcm!.takeBytes())).trim();
await _send(text);
Waiting for _micDone is not ceremony: stop() closes the stream, and the last chunk lands in the buffer just before it ends. Read the buffer on stop() alone and you lose the tail of the sentence.
_send takes the transcript the way it takes typed text. The model never hears audio; it gets words, the same as before.
Three platforms, three places:
android/app/src/main/AndroidManifest.xml: ios/Runner/Info.plist: an NSMicrophoneUsageDescription stringmacos/Runner/Info.plist, andcom.apple.security.device.audio-input set to true in both DebugProfile.entitlements and Release.entitlementsWatch out: Miss the macOS entitlement and nothing fails. The sandbox hands the app silence, the recognizer transcribes silence, and you get an empty transcript — which reads exactly like "I did not speak loudly enough".
cd ../step_02_hear
flutter run
The first run downloads about 2.7 GB. Then tap the mic, ask a question, tap again. The transcript appears as your message, and the model answers in text.
Good to know: Transcribing near-silence can produce words that were never said — that is a property of speech models, not a bug in the app. If you get a phantom question, you probably tapped stop before you spoke.
flutter pub add just_audio path_provider
flutter_gemma_speech already contains text-to-speech. What you need is something to play its output, and a place to put the file the player reads.
sttBackends: [const LiteRtSttBackend()],
ttsBackends: [const LiteRtTtsBackend()],
The voice is Inflect-Nano-v2: English, the fastest of the three the package ships, and about 36 MB on disk. Its own two networks are 8 MB; the rest is the pronunciation data it shares with Matcha-TTS, which the installer fetches from the Matcha repository on its own. You give it one URL:
abstract final class Inflect {
static const baseUrl =
'https://huggingface.co/sasha-denisov/inflect-nano-v2-litert/resolve/main/';
}
await FlutterGemma.installTts()
.fromNetwork(Inflect.baseUrl)
.ofType(TtsModelType.inflect)
.install();
final tts = await FlutterGemma.getActiveTts();
The same install-on-every-launch as Step 2, for the same reason.
synthesize returns bare samples — numbers, with nothing saying how fast to play them. A player needs to be told: one channel, so many samples per second, 16 bits each. That is all a WAV header is, and lib/wav.dart writes it:
Uint8List wavFromPcm16(Uint8List pcm, {required int sampleRate}) {
const channels = 1;
const bitsPerSample = 16;
const blockAlign = channels * bitsPerSample ~/ 8;
final header = ByteData(44)
..setUint32(0, 0x52494646) // "RIFF"
..setUint32(4, 36 + pcm.length, Endian.little)
..setUint32(8, 0x57415645) // "WAVE"
// ...
..setUint32(24, sampleRate, Endian.little)
// ...
..setUint32(40, pcm.length, Endian.little);
// header, then the samples
}
Watch out: Use tts.sampleRate, never the 16 kHz you recorded at. Inflect speaks at 24 kHz. Write 16000 into the header and the voice plays at two thirds of its speed, a fifth lower — recognisably the app, and recognisably wrong.
Once the whole reply is in:
Future<void> _speak(String text) async {
final tts = _tts;
if (tts == null || text.trim().isEmpty) return;
final pcm = await tts.synthesize(text);
if (pcm.isEmpty || !mounted) return;
final dir = await getTemporaryDirectory();
final file = File('${dir.path}/reply.wav');
await file.writeAsBytes(wavFromPcm16(pcm, sampleRate: tts.sampleRate));
await _player.setFilePath(file.path);
unawaited(_player.play());
}
play() completes when playback ends, so it is not awaited: the composer unlocks while the answer is still being read out. A reply with nothing pronounceable — an emoji — comes back as zero bytes, and there is nothing to play.
And when you start recording, stop the player first. Otherwise the microphone records the app's own voice along with yours:
await _player.stop();
Ask "What is the capital of France?". The answer appears, then you hear it.
Listen to how it says it. Gemma writes its answers for a screen, and a screen answer has bold text and lists in it — The capital of France is **Paris**. The voice copes with the asterisks, but the answer is still shaped for reading. The next step fixes that at the source.
Everything the model writes is now heard, so tell it:
const _spokenStyle =
'You are a voice assistant. Your reply is read aloud by a speech '
'synthesizer, so answer in one or two short sentences of plain spoken '
'English. No markdown, lists, emoji or code.';
final chat = await inference.createChat(
modelType: widget.model.modelType,
systemInstruction: _spokenStyle,
maxOutputTokens: 128,
);
The same question now comes back as The capital of France is Paris. A small model takes an instruction like this literally, which is why it says what to leave out as well as what to do.
Good to know: maxOutputTokens caps the reply. maxTokens on getActiveModel is the whole context window — prompt, history and reply together — and is not a length limit at all. A spoken answer that runs past two sentences is one nobody waits for.
Steps 2 and 3 wired recognizer → chat → synthesizer by hand. VoiceSession does that, in order, and reports what happened as events:
_session = VoiceSession.fromChat(
recognizer: stt,
chat: chat,
synthesizer: tts,
);
It owns none of the three: the page still closes them. It does not own the microphone or the speaker either. What it owns is the order — one turn at a time, and a stop that reaches every stage. Step 5 is where that last part pays off.
A recorded question now runs through one call:
await for (final event in _session!.runTurn(audio)) {
switch (event) {
case VoiceTranscriptEvent(:final text):
// show what the user said
case VoiceReplyTextEvent(:final chunk):
// append to the reply bubble
case VoiceReplyAudioEvent(:final pcm, :final sampleRate):
await _play(pcm, sampleRate);
case VoiceTurnCompleteEvent(:final transcript):
// an empty transcript means nothing was heard
case VoiceTurnInterruptedEvent():
case VoiceErrorEvent():
break;
}
}
The switch is exhaustive over a sealed type, so an event the package adds later fails to compile instead of being silently ignored.
Near-silence transcribes to nothing, and the session ends the turn right there without asking the model anything — a turn with an empty transcript is complete, not failed.
Watch out: A stage that fails — transcribe, generate, synthesize — arrives as an error on the stream, not as an event. VoiceErrorEvent is reserved and not emitted in this release. Wrap the loop in try/catch, or a failed turn looks like a turn that never ended.
The typed path stays as it was: a typed question goes straight to the chat and through _speak. Both land in the same conversation.
Talk to it. Transcript, reply and voice now come out of one call, and the answers sound like answers.
Two things are still wrong. You wait for the whole answer to be written and then synthesized before hearing a word. And once it starts talking, you cannot stop it.
_session = VoiceSession.fromChat(
recognizer: stt,
chat: chat,
synthesizer: tts,
streamAudio: true,
);
With streamAudio, the session synthesizes the reply clause by clause as the tokens arrive. You get several VoiceReplyAudioEvents with isFinal: false, then one empty one with isFinal: true that only says "that was all".
On an M4 Pro the first audio arrived about 0.6 s after the question finished — before the model had written its second sentence.
Several clips means a queue: they must play one after another, never on top of each other, and a stop must silence the one playing and every one still waiting. lib/speaker.dart is that queue:
void enqueue(Uint8List pcm, int sampleRate) {
if (pcm.isEmpty) return;
final generation = _generation;
_tail = _tail.then((_) async {
if (generation != _generation) return;
// write the clip to a file, then:
await _player.stop();
await _player.setFilePath(file.path);
if (generation != _generation) return;
await _player.play();
}).catchError((Object error) => onError(error));
}
Future<void> stop() async {
_generation++;
await _player.stop();
}
Each clip is chained onto the one before it. stop() bumps a generation counter, so a clip queued before the stop sees the new number and skips itself. The check runs a second time right before play(): a stop can land while the file is being loaded, and without it a clip from before the barge-in plays over the microphone that has just opened.
A clip that cannot be written or played goes to onError, which the page shows. Dropping it quietly would leave a reply on screen that was never heard, with nothing to say why.
Watch out: The _player.stop() before each clip is load-bearing. After a clip ends, just_audio still reports playing, and play() on a playing player returns at once without playing anything. Stopping first puts it back where play() means "play this".
A tap on the mic while the assistant is talking now stops it and starts listening:
Future<void> _toggleMic() async {
if (_listening || _opening) {
await _stopListening();
return;
}
_opening = true;
try {
await (_micStarting = _interruptThenListen());
} finally {
_opening = false;
}
}
Future<void> _interruptThenListen() async {
if (_turn != null) await _interrupt();
await _startListening();
}
Future<void> _interrupt() async {
await _speaker.stop();
await _session?.interrupt();
await _turn;
}
Silence first: the speaker stops in a millisecond, while the session has to wait for generation to notice it was asked to stop. interrupt() stops the model, drops any clause not yet synthesized, and completes once the turn has ended with a VoiceTurnInterruptedEvent. The mic button stays enabled during a voice turn for exactly this reason.
Opening the microphone takes a moment — a permission prompt the first time, the recorder starting every time — and _listening turns true only at the end of it. A second tap in that gap is a stop, not a second start: _opening routes it to _stopListening, which waits for the whole opening, interrupt included, before it stops.
The interrupted event carries what the model had written when it was stopped:
case VoiceTurnInterruptedEvent(:final partialReplyText):
// mark the bubble "(interrupted)"
And the chat keeps it. After interrupting "Tell me three facts about the moon", the history holds the question and the half-written answer, so the model knows what it had already said. Your next question is answered on top of that.
Good to know: This needs flutter_gemma_litertlm 1.8.1 or newer. Before it, a .litertlm chat that was stopped mid-reply answered every later message with nothing. The package now rebuilds the conversation from the chat's history on the first turn after a stop. That history is replayed as text: an image or a recording sent in an earlier turn is not part of it, so ask about it again with the image attached.
Ask for three facts about the moon, and tap the mic while it is on the first one. It stops mid-word, listens, and answers the next question in full.
An assistant that can only talk is a chat with a speaker. This step lets it do two things on the device: read the clock, and set a timer that the app keeps and announces when it runs out.
lib/tools.dart:
const clockTool = Tool(
name: 'get_current_time',
description:
'Return the current local time on this device. Use this whenever the '
'answer depends on what time it is now.',
parameters: {'type': 'object'},
);
const timerTool = Tool(
name: 'set_timer',
description: 'Start a countdown timer on this device.',
parameters: {
'type': 'object',
'properties': {
'minutes': {'type': 'number', 'description': 'Length in minutes.'},
},
'required': ['minutes'],
},
);
const toolbox = [clockTool, timerTool];
The description is the only thing telling the model when to call, so it is written as instruction. The clock takes no arguments and still declares parameters with a type — without it, the model reads the declaration as cut off.
Both tools are local and instant, so the assistant keeps working in airplane mode.
Map<String, dynamic> run(FunctionCallResponse call) => switch (call.name) {
'get_current_time' => {'time': _clock(DateTime.now())},
'set_timer' => _setTimer(call.args['minutes']),
_ => {'error': 'There is no tool called ${call.name}.'},
};
A tool is a Dart function from the arguments the model wrote to the map it will be shown next. Check those arguments like any other input — the model can write "five" or 0, and can name a tool that does not exist. Answer that with an error in the result, not an exception: the model reads it and can say what went wrong.
When a timer runs out, nobody asked a question, so there is no turn to answer in. The app says it directly:
Future<void> _timerDone(num minutes) async {
final whole = minutes == minutes.roundToDouble() ? minutes.round() : minutes;
final length = whole == 1 ? 'one minute' : '$whole minute';
final text = 'Your $length timer is done.';
if (!mounted) return;
_say(text);
await _speak(text);
}
final chat = await inference.createChat(
modelType: widget.model.modelType,
systemInstruction: _spokenStyle,
maxOutputTokens: 128,
tools: toolbox,
supportsFunctionCalls: true,
);
_session = VoiceSession.fromChat(
recognizer: stt,
chat: chat,
synthesizer: tts,
streamAudio: true,
onToolCall: _tools.run,
);
onToolCall is required once the chat has tools. Without it a tool call has nothing to run, and the session refuses to start. The loop that sends the result back and lets the model answer is the SDK's, not yours.
The typed path needs the same loop — the model may call a tool whichever way the question arrived:
await for (final chunk in chat.generateChatResponseWithTools(
onToolCall: _tools.run,
)) {
// ...
}
Ask "What time is it?". The model calls get_current_time, reads the result, and says it: It is currently twelve fifty-five.
Run it a few times and sometimes you get It is currently 2:27 PM. instead. The spoken-style instruction from Step 4 nudges the model toward numbers the way they are said, and a small model does not follow a nudge every time. The words are what the voice reads best: played back to the recognizer, "twelve fifty-five" comes back as "1255", while "2:27 PM" comes back as "two, 27th M".
Then ask it to "set a timer for one minute", and a minute later the app tells you it is done.
Everything above runs on the device. Prove it: switch on airplane mode and restart the app.
install() calls download nothing — they only re-activate.What needs the network is the first run's download, and nothing else.
SttModelType.whisper understands 99 languages, and getActiveStt(language: 'de') picks the output one; Qwen3-TTS is the multilingual voice. Both are larger and slower than the models used here.ofType away; Matcha is the other English voice the package ships.VoiceSession.custom with your own responder puts it behind the microphone.