4.4M params

ThinkSpark-4.4M

A small model that fills the silence after someone stops talking — so production calls never sit in dead air.

4,445,809 parameters. Smaller than a JPEG. A byte-level dual-encoder that predicts the human backchannel / thinking sound a voice agent should murmur in the gap between the user’s speech (STT) and the agent’s reply (TTS) — hmm, अच्छा, એક સેકન્ડ, ええと — in the right language, native script, register and emotion.

Every voice product has the same awkward beat: the customer stops talking, the system is still thinking, and the line goes quiet. People hang up. They think they got a scam IVR. They press zero. ThinkSpark is not the main voice. It is not the LLM. It is the presence layer until the answer is ready.

No GPU at inference. 2–8 ms on CPU. 259-row byte vocab, zero OOV across every script. Ships as ONNX into the Kupe voice worker with thinking_sounds.mode = "auto". We use it in production so a ledger lookup or a calendar check never sounds like a drop.

4.4M

parameters

2–8 ms

CPU inference

22

languages

259

byte vocab

Task

What it predicts

Given the user’s current utterance (input, primary) plus the past conversation (context, any language or mix), it outputs five heads. A curated filler dictionary then samples the actual spoken spark for (language, intent, filler_type) — or silence when the model says no_filler.

Intent

9 agent reactions

thinking, clarify, acknowledge, disagree, react_positive, empathize, soothe, impatient — and silence. Silence is first-class: many turns need nothing.

Language

22 native scripts

Hindi, Marathi, Bengali, Gujarati, Punjabi, Tamil, Telugu, Kannada, Malayalam, Odia, Assamese, Urdu, Hinglish, English, Spanish, French, German, Portuguese, Japanese, Mandarin, Arabic, Russian.

Register

How they talk

formal, casual, or urban_mixed. The same intent in a bank collections call should not sound like a friend on WhatsApp.

Emotion

Tone of the moment

neutral, warm, curious, concerned, cheerful, apologetic, firm, playful, tired, anxious — colour on the spark, not a second script.

Filler type

The shape of the spark

sound (hmm), word (अच्छा), sound_word (hmm अच्छा), words (एक सेकंड), or none. The dictionary samples the spoken form after the heads fire.

hiहिन्दीmrमराठीbnবাংলাguગુજરાતીpaਪੰਜਾਬੀtaதமிழ்teతెలుగుknಕನ್ನಡmlമലയാളംorଓଡ଼ିଆasঅসমীয়াurاردوhi_enHinglishenEnglishesEspañolfrFrançaisdeDeutschptPortuguêsja日本語zh中文arالعربيةruРусский

Architecture

How the diagram works

ThinkSpark is a dual-encoder, not one flat transformer. The two inputs play different roles, so they are encoded separately, then fused with cross-attention. Read the figure top to bottom — it is the whole forward pass.

ThinkSpark

ultra-lightweight dual-encoder

~4.4M params · 259-byte vocab · CPU 2–8 ms · 22 languages · zero OOV

Ultra light

smaller than a JPEG

Input · primary

last user utterance

STT text · 96 bytes · any script

Context · modulates

past conversation

multi-turn · 384 bytes · any mix of languages

Tokenizer

byte-level tokenizer · UTF-8 bytes · vocab 259 · zero out-of-vocabulary

Devanagari Tamil Gujarati kana Arabic Latin emoji — same tiny embedding table

Input encoder

5 transformer layers

d_model 192 · 6 heads · GELU FFN ×3 · shared byte embedding · pre-norm

Context encoder

3 transformer layers

same width · lighter stack · independent weights · never overwrites the input signal

Cross-attention fusion · 2 layers

input tokens query the context · context is key / value

same “okay” yields an apologetic spark after a complaint · a curious one in small talk

Intent

9 agent reactions

Language

22 native scripts

Register

formal · casual · mixed

Emotion

tone of the moment

Filler type

sound · word · none

Spark · sampled from filler dictionary

hmm · अच्छा · એક સેકન્ડ · ええと · “got it” · or silence

then TTS, while the LLM writes the real reply

Dual-encoder · 4.4M params · light theme, Kupe tokens

STT text → ThinkSpark + context → spark (“हम्म, एक सेकंड…”) → TTS → (LLM answer streams in)

01

Two inputs, two jobs

Input is the last user utterance — the primary signal, 96 bytes of STT text in any script. Context is the past conversation — up to 384 bytes, any mix of languages. Context modulates. It never overwrites the input. The same “okay” after a repeated refund complaint becomes an apologetic murmur; during small talk it stays curious.

02

Byte tokenizer, zero OOV

Every script is UTF-8 bytes against a 259-row embedding. Devanagari, Tamil, Gujarati, kana, Arabic, Latin, emoji — same tiny table. No subword vocab to maintain, no unknown tokens when a caller code-switches mid-sentence.

03

Dual encoders

The input encoder is five transformer layers (d_model 192, 6 heads, GELU FFN ×3). The context encoder is three layers at the same width, independent weights. Context stays a lighter stack so it never drowns the last thing the caller said.

04

Cross-attention fusion

Two fusion layers. Input tokens query the context; context is key and value. The pooled representation is anchored on the input and only coloured by the conversation so far — exactly the “input is more important, context adjusts it” behaviour a live call needs.

05

Five heads, then a spark

Shared trunk, five classification heads. Predicted (language, intent, filler_type) samples a real surface form from filler_dictionary.json — or silence when the model says no_filler. That spark goes straight to TTS while the LLM writes the real reply. Editing a language’s fillers is editing the lexicon. No retraining.

Input · primary

Last user utterance

The line they just finished. Example: “अरे यार फिर से वही दिक्कत”. 96 bytes. Any script.

Context · modulates

Past conversation

A real transcript, not an English note. Hindi turns, an English agent line, empty on a cold start — the byte tokenizer reads all of it. 384 bytes.

Outputs

What comes out

Each prediction is a spark plus the five heads. The spark is what TTS speaks immediately. The heads are what you log. When intent is silence, the spark is empty and the line stays quiet on purpose.

Hindi · soothe

माफ़ कीजिए

Input

अरे यार फिर से वही दिक्कत

Context

User: रिफंड नहीं आया
Agent: sir I am checking
User: कितनी बार बोलूँ

Heads

intent
soothe
language
hi
register
casual
emotion
apologetic
filler_type
words

The model-card example. Repeated complaint + an English agent turn → a Hindi apology, not another “hmm”.

English · acknowledge

got it

Input

I need to move Thursday to Friday evening.

Context

User: can we reschedule
Agent: sure, which day works

Heads

intent
acknowledge
language
en
register
casual
emotion
neutral
filler_type
word

A short word, not a thinking sound. The caller already said the slot — presence is enough.

Tamil · thinking

ஒரு நிமிடம்

Input

என் EMI இந்த மாசம் என்ன?

Context

User: account number 4481
Agent: one moment

Heads

intent
thinking
language
ta
register
formal
emotion
neutral
filler_type
words

Ledger lookup. Tamil caller, Tamil spark — not an English “um” dropped into a Tamil conversation.

Japanese · thinking

えっと

Input

来週の火曜、空いてますか?

Context empty · cold start

Heads

intent
thinking
language
ja
register
casual
emotion
curious
filler_type
sound

Cold start, empty context. A native thinking sound while the calendar tool runs.

Hindi · silence

· (silence)

Input

हाँ, वो वाला ही।

Context

Agent: EMI 4,280 rupees, due on the 5th
User: confirm that

Heads

intent
silence
language
hi
register
casual
emotion
neutral
filler_type
none

The answer is ready. no_filler is a real class — many turns should stay quiet.

How to use

Turn it on, or pull the weights

Production Kupe agents already run this path. In the hub, set Thinking sounds to Auto. The worker downloads ONNX + labels + filler dictionary from Hugging Face at warmup, predicts on CPU in a few milliseconds, and speaks the spark on TTS while the LLM writes. If the model cannot load, Auto degrades to silence — a turn is never blocked by it.

01 / Hub

Open the agent → Thinking sounds → Auto. Same control as sounds / words / off.

Open the hub

02 / Realtime API

PATCH the agent with thinking_sounds.mode auto. The hub path is the production path.

Realtime

03 / Weights

Snapshot the HF repo and run ThinkSparkPredictor locally. ONNX for the voice worker.

Hugging Face

Agent config

{
  "thinking_sounds": { "mode": "auto" }
}

Modes: off silence · sounds static hesitation · words static acknowledgement · auto ThinkSpark. In auto, the LLM is instructed not to open with a filler — so the caller does not hear it twice.

Python · Hugging Face

from huggingface_hub import snapshot_download
from thinkspark.infer import ThinkSparkPredictor

local = snapshot_download("anuj-inavlabs/ThinkSpark-4.4M")
pred = ThinkSparkPredictor(local, f"{local}/filler_dictionary.json")

pred.predict(
    input_text="अरे यार फिर से वही दिक्कत",
    context="User: रिफंड नहीं आया\nAgent: sir I am checking\nUser: कितनी बार बोलूँ",
)
# -> {'spark': 'माफ़ कीजिए', 'intent': 'soothe', 'language': 'hi', ...}

CLI · from the repo

python scripts/04_infer.py --ckpt artifacts/thinkspark/best \
  --input "अरे यार फिर से वही दिक्कत" \
  --context $'User: रिफंड नहीं आया\nAgent: sir I am checking\nUser: कितनी बार बोलूँ'

# interactive REPL — type the user's line; prefix a line with @ to set context
python scripts/04_infer.py --ckpt artifacts/thinkspark/best

Train, export ONNX, and push with the kupe-thinkspark pipeline. Data lives at anuj-inavlabs/kupe-thinkspark (38,338 clean rows). Native-speaker QA is recommended for urban / code-mixed and low-resource fillers before you ship a new language.

Test metrics

Held-out numbers

Intent acc / top-2 / macro-F1

0.440 / 0.640 / 0.340

Language accuracy

0.967

Emotion / filler_type accuracy

0.444 / 0.536

Parameters

4,445,809 (~4.4M)

Inference

2–8 ms on CPU, no GPU

Vocab

259-row byte table, zero OOV

Trained with focal loss and class-balancing (intent is sharply skewed). Native-script language is pinned by a script guard at inference. Generated by kupe-thinkspark; data via Sarvam sarvam-105b. Apache-2.0.

ThinkSpark-4.4M · Kupe