Latest release

Qwen3.8-Flash-Next 141B-A6B Niwaki v4.0 · lattice-coded experts

Qwen3.8-Flash-Next at 141B total and about 5.9B active parameters, its routed experts stored as E8 lattice codes at about 3, 1.4 or 1 bit per weight: 44.8 GB on disk, the same as v2.4, for 64 GB Macs. In that space it keeps more of the model's experts than v2.4 — 17,259 of 24,576 — and reads 10% lower perplexity. The task average reaches 0.794 against the unpruned model's 0.759, the program-state behaviours we test stay at the unpruned model's level, and twenty languages sit within 0.96–1.17× of its perplexity.

• 44.8 GBon disk, the same as v2.4; from 111.5 GB for the 4-bit reference
• −10%WikiText-2 perplexity against v2.4 at the same size (5.70 against 6.36)
• 0.794task average, against 0.759 for the unpruned model and 0.762 for v2.4
• 98.0%of 2,000 fresh two-step arithmetic problems, as the unpruned model

• Niwaki v4.0 and v5.0
• earlier Niwaki generations
• the unpruned model

Perplexity over 64 × 2048-token windows; task average over arc_easy, hellaswag, piqa, winogrande and boolq, zero-shot, 500 examples each, the same examples for every row; languages on FLORES-200 devtest (English, Spanish, French, German, Russian, Chinese, Japanese, Korean, Arabic and Hindi), 1.00 meaning the unpruned model's perplexity; the program-state suite (variable tracking, distractor lines, two-step arithmetic, multi-hop lookups) scored by forced choice on two held-out seeds, and the arithmetic figure on 2,000 freshly generated problems scored the same way. The task average sits above the unpruned model's on the strength of boolq, winogrande and arc_easy; on hellaswag and piqa v4.0 stays below it. v2.4 stays lower on Hindi (0.97 against 1.12) and Malayalam; on the other eighteen of the twenty languages we measure, v4.0 reads lower. The best 2-bit cast of the unpruned model (routed experts and n-gram table at 2-bit) is 10.5 GB larger than v4.0 and behind it on both perplexity and task average. In MLX, v4.0 decodes 2–3% slower than v2.4, and its prompt throughput on 2,048 tokens is 9–13% lower. Protocols and the full tables are on the model cards.

In llama.cpp

The v4.0 file stores the same experts as the MLX weights at higher precision, Q3_K and IQ4_NL, which makes it 17 GB larger; on identical tokens it reads 9% lower perplexity than they do. On an M4 Max with Metal (llama-bench, in one session with v2.4's UD-Q3K file) it runs at 506 tokens/s on a 512-token prompt, 396 on a 2,048-token prompt and 26 in decode, against 488, 417 and 26 for v2.4. On a Mac it holds 42.0 GiB on the GPU at a 2,048-token context, within a 64 GB Mac's default GPU limit; its n-gram table stays in CPU memory, read from the file as tokens need it. The 23GB file is the GGUF build of the 19 GB model below and holds 21.8 GiB on the GPU. Like the v2.4 files, these need the neopolita-llama.cpp fork: stock llama.cpp cannot represent this model's layout yet.

• MLX weights ↗44.8 GB · Apple silicon, 64 GB
• GGUF build ↗61.7 GB · llama.cpp

For 32 GB Macs: v4.0 · 19 GB

The v4.0 model sized for 32 GB Macs: its routed experts are stored as E8 lattice codes at about 2 bits per weight, and recovery training follows. Its 14.4 GB n-gram table stays on disk and is read as tokens need it, so the model holds about 19 GB in memory although the download is 33.3 GB. Against v2.4's 102B, the smallest v2.4 release, it reads lower perplexity and KL divergence, a higher task average and an equal or higher score on every program-state behaviour.

• 19.1 GiBpeak on a 2,048-token prompt, under a 32 GB Mac's default GPU limit of 21.3 GiB
• −17%WikiText-2 perplexity against v2.4's 102B (6.51 against 7.84)
• 0.765task average, against 0.738 for v2.4's 102B and 0.759 for the unpruned model
• 92.9%of 2,000 fresh two-step arithmetic problems, against 98.0% for the unpruned model

• MLX weights ↗33.3 GB download · Apple silicon, 32 GB
• GGUF build ↗40.2 GB · llama.cpp

For 24 GB Macs: v5.0 · 15 GB

The 19 GB model with its routed experts stored as trellis codes at about 1.5 bits per weight instead of E8 lattice codes at about 2: the same experts and everything else, in 11.9 GB of expert storage instead of 15.7 GB. It holds about 15 GB in memory against a 29.5 GB download, so a 24 GB Mac runs it with prompts up to about 8K tokens. Against the 19 GB model it reads 4–6% higher perplexity and a little lower variable tracking; against v2.4's 102B, lower perplexity and KL divergence and a higher task average. Two-step arithmetic matches the 19 GB model when it thinks first (60 of 60 word problems, against 58) and falls behind when it answers directly (42 against 50). There is no GGUF build: llama.cpp does not read trellis-coded experts.

• 15.8 GiBpeak on an 8,192-token prompt, under a 24 GB Mac's default GPU limit of about 16 GiB
• −12%WikiText-2 perplexity against v2.4's 102B (6.90 against 7.84)
• 0.760task average, against 0.738 for v2.4's 102B and 0.759 for the unpruned model
• 92.9%of 2,000 fresh two-step arithmetic problems, as the 19 GB model

• MLX weights ↗29.5 GB download · Apple silicon, 24 GB

Earlier generations

• v2.4 · Qwen3.8-Flash-Next 119B-A5B · 3-bit 44.8 GB · perplexity 6.36 · task average 0.762 MLX ↗ · GGUF ↗
• v2.4 · Qwen3.8-Flash-Next 102B-A5B · 3-bit 37.5 GB, fits a 48 GB Mac · perplexity 7.84 · task average 0.738 MLX ↗ · GGUF ↗
• First generation · Qwen3.8-Flash-Next 113B-A5B · 3-bit 42.2 GB · perplexity 7.08 · task average 0.757 MLX ↗ · GGUF ↗
• First generation · Qwen3.8-Flash-Next 99B-A5B · 3-bit 36.7 GB, fits a 48 GB Mac · perplexity 7.88 · task average 0.733 MLX ↗ · GGUF ↗