Qwen 3.8 27B Just Hit 200 Token/s on Mac M5 (Lithos Metal)

LithosAI says Qwen 3.8 27B runs at more than 200 tokens a second on an M5 Max, and its own measured run five days earlier works out to about 80. lithos-metal is a new open-source Metal engine for Apple silicon. It uses DSpark speculative decoding: a 1.1 GB drafter guesses seven tokens, and the big model checks all of them in one read of its weights. That's how a 40-core M5 Max, which tops out near 38 tokens a second on the 16 GB 4-bit Qwen 3.8 27B file (614 GB/s ÷ 16 GB), can go past its own memory ceiling. The README's 168 tokens a second is a projection that assumes six of those seven guesses land every round. The team's own four-prompt test averaged two and a half, at 12.47 ms of GPU time per token. The hit rate depends on what you ask for. One 40-core M5 Max owner measured lithos-metal at 59 tokens a second on code against oMLX's 55, and 34 on English against 38. For a local AI coding agent, reading matters more than writing: an AgentPerf-Local session reads about 196,000 new tokens for every 31,000 it writes, and the v0.1.2 release Homebrew installs today reads prompts about 2.5x slower than oMLX. Two prefill fixes are already merged on main but not released yet. Which Mac this applies to: the tuned path exists only for the M5 Max with the 40-core GPU. The 32-core M5 Max (460 GB/s, 36 GB), every M4 Max, and the M5 Pro on large models run an unmeasured fallback, and there's no M6 Mac mini backend yet. If you run Qwen 3.8 27B on a Mac through MLX, Ollama, oMLX or Splash, this tells you whether switching engines is worth it this month. WHAT THIS VIDEO COVERS Why 614 GB/s over a 16 GB file caps plain decoding near 38 tokens a second Where 168, 127 and 200 come from, in the README's own words The measured run: 2.5 guesses accepted per round, about 80 tokens a second Four Mac headlines in nine weeks (30.5, 70, 144, 200) and the three different things they counted Code vs English on the same chip: 59 vs 34 The agent bill: six tokens read for every one written Which Macs get the fast path, and who should install it now CHAPTERS 0:00 The chai and the 200 headline 1:35 The 38 a second ceiling 3:11 The guessing trick behind 200 5:35 Where the 200 comes from 7:27 Their own run: about 80 8:51 Every Mac headline doubles 10:33 Code drafts well, English doesn't 12:46 The agent bill: reading vs writing 15:09 Which Mac, which version 17:06 Install it? Three questions LINKS FROM THE VIDEO lithos-metal repo: https://github.com/lithos-ai/lithos-metal lithos-metal README (read this before the blog): https://github.com/lithos-ai/lithos-metal#readme The measured study (DSpark round + four-prompt run): https://github.com/lithos-ai/lithos-metal/blob/46b2bc4cda57826c072d94afe90a58633aaeb6d9/docs/research/m5max-27b-dspark-refinement.md LithosAI blog (Oct 8, 2026): https://www.lithosai.com/blog/lithos-metal SOURCES Backend comparison and projection method: https://github.com/lithos-ai/lithos-metal/blob/46b2bc4cda57826c072d94afe90a58633aaeb6d9/docs/research/m5max-27b-backend-verification.md Prefill fix, Oct 7: https://github.com/lithos-ai/lithos-metal/pull/1 Prefill fix, Oct 11: https://github.com/lithos-ai/lithos-metal/pull/28 Greedy divergence report (M3 Max): https://github.com/lithos-ai/lithos-metal/issues/54 DSpark paper (DeepSeek): https://arxiv.org/abs/2607.05147 Qwen3.8-27B model card: https://huggingface.co/Qwen/Qwen3.8-27B MacBook Pro specs (614 / 460 / 307 GB/s): https://www.apple.com/macbook-pro/specs/ Plain 4-bit on M5 Max, 30.5 tok/s: https://www.reddit.com/r/LocalLLaMA/comments/1vokoir/ DFlash 2 at 70 tok/s: https://x.com/i/status/2089836737132650504 Splash at 144 tok/s: https://x.com/i/status/2101100749623341513 MLTF 200+ at four requests: https://www.reddit.com/r/oMLX/comments/1wvrb57/ RTX 3090 381 tok/s on document copying: https://x.com/i/status/2108399085514371078 mlx-dspark math / code / chat speedups: https://www.reddit.com/r/LocalLLaMA/comments/1vokrcy/ Thinking xhigh vs off on M5 Max: https://www.reddit.com/r/LocalLLaMA/comments/1w1vbal/ AA-AgentPerf-Local: https://artificialanalysis.ai/articles/aa-agentperf-local WATCH NEXT Qwen 3.8 27B Just Got 2× Faster on Mac Same Hardware: https://www.youtube.com/watch?v=qZTmBhKz7b0 Why Your Mac Is Slow at Local AI (And It's Not the Chip): https://www.youtube.com/watch?v=x_qmqqp6d7s How Fast Is Local AI on Your Mac? (Every Chip, M1 to M6): https://www.youtube.com/watch?v=WoGwVXFmWzU I Tested Every Qwen3.8-27B Quant: Here's the Best One For Your GPU: https://www.youtube.com/watch?v=hg26j0erUSo You Don't Need Frontier Models Anymore (Qwen 3.8 27B + DeepSeek Harness): https://www.youtube.com/watch?v=3DVRznjCIS8 The End of VRAM-Bottlenecked LLMs: Qwen3.8-Flash-Next: https://www.youtube.com/watch?v=MPrQA-M4Zn4 Don't Buy a Mac Mini For Local AI (Do This): https://www.youtube.com/watch?v=IRyiP_U7RuA Can a Mac Mini Replace your $200 AI Subscription?: https://www.youtube.com/watch?v=3R7yba577uA #qwen #localllm #applesilicon