InnerThink

Orange mascot with duct tape over its mouth — thinking that never becomes words.
The token bill

You pay for every word it thinks.
Almost none of them are for you.

Qwen3-8B · thinking mode 0tokens
You A shirt is $40 and it's 25% off. What do I pay?
Thinking
Reply $30. ↑ the only part you read
InnerThink · same question 0tokens
You A shirt is $40 and it's 25% off. What do I pay?
Thinking thinking silently 6 latent steps · 0 tokens decoded
Reply $30. ↑ same answer, 8× cheaper
Thinking tokens
—→0
Total generated
—→14
The answer
$30 = $30
Token bill
—
Illustrative single turn. The thinking counter counts exactly the words on screen; a real Qwen3-8B trace on this question is typically longer, not shorter. Compression range for latent CoT: 3.1× (CODI, EMNLP 2025).
Where the thinking lives

A thought either becomes a word,
or it stays in the hidden layers.

Every other LLM · explicit CoT 0reasoning tokens
3 reasoning steps · 27 tokens decoded, emitted, and read back in
decoded into words, fed back in Question Answer 40 / 4 = 10

Every step leaves the network, is decoded into words, and is fed back in as input. You are billed for each one.

InnerThink · latent CoT 0reasoning tokens
3 reasoning steps · 3 forward passes, 0 tokens decoded
Hidden layers Question Answer

The last hidden layer loops straight back to the first. The forward passes still cost compute — they just never cost a token.

Five layers stand in for 36 and three reasoning steps for six: CODI-Qwen3-8B runs six latent steps through the full transformer stack, with the recurrence being the last hidden state projected through a 2-layer MLP + LayerNorm. Counts are reasoning tokens only, at ~9 per illustrated step — both sides still emit the answer, and a latent step still costs a forward pass. What it does not cost is a token.
How it learns

The words teach the vectors.
Then the words are thrown away.

Self-distillation: one network runs both tasks. The student never copies the teacher's words — it copies what those words did to the network inside.

One network shared weights Teacher task · explicit CoT question 40 / 4 = 10 40 − 10 = 30 the answer is : $30 cross-entropy on every word · 27 tokens of reasoning Student task · latent CoT question bot eot the answer is : $30 cross-entropy on the answer only · 0 tokens of reasoning Teacher · hidden state at this token Student layer 1 → 36 L1 at every layer Writing the reasoning out shifts the teacher's whole stack at this token. The student has to reproduce that shift without writing anything. stop-gradient on the teacher Objective ℒteacher CE on the words ℒstudent CE on the answer ℒdistill L1 across all layers At inference Only the student task runs. The teacher existed to produce one hidden state for the student to match. It is never executed when the model answers.
LoRA r=128, α 32 on q,k,v,o,gate,up,down 0.35 B trained / 8.2 B frozen ≈388k GSM8k-Aug-NL + CommonsenseQA 5 epochs · 8× H100 · lr 2e-4
Method and figures from Shen et al., CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation, EMNLP 2025, and the CODI-Qwen3-8B model card. Adapter size computed from the Qwen3-8B config at rank 128.
Proof at scale

Every answer,
already computed.

GSM8K · seed — 0/ —
Cached questions
0
Direct accuracy
0%
Latent accuracy
0%
Verbal accuracy
0%
Aggregate inference time avg visible tokens saved 0

Three paths, same — questions.

Directno reasoning — answer only
— avg  ·  — total
LatentInnerThink — 6 vectors, 0 tokens
— avg  ·  — total
Verbalexplicit CoT — thinking mode
— avg  ·  — total
Simulated batch summary for the live demo — the point being made is that answers are cached ahead of time, not re-run on stage. Wire BATCH in the script to a logged eval before presenting these as measured numbers. Bar widths are proportional to avg latency, computed at runtime, not hand-set.

InnerThink

Orange mascot with duct tape over its mouth — thinking that never becomes words.