Why Your Mac Is Slow at Local AI (And It's Not the Chip)

Local AI on a Mac feels slow for a reason the tokens-per-second counter never shows: your local LLM re-reads the whole prompt every turn. An M5 Max MacBook Pro with 128GB of unified memory spent 8 hours making 15 edits with a coding agent, and the chip was never the problem. Every "why is local AI slow on my Mac" video lands on memory bandwidth, and that part is real. A Mac mini M6 moves 170 GB/s, an M5 Max moves 614 GB/s, and the M5's neural accelerators made prompt reading about 3.6x faster than the M4 Max on llama.cpp's own benchmarks. But bandwidth only sets how fast the model writes. Before the first word, it has to read everything you sent it, and in an agent loop that reading is most of the wait. One M1 Max showed 57 tokens per second in LM Studio while the real speed at 8,500 tokens of context was 3. The fix is the KV cache, and specifically prefix caching. On the same M5 Pro, in the same session, with the same Qwen3.8-27B model, the first word took 6.1 seconds when the start of the prompt matched the cache and 268 seconds when it didn't. That's 44x, which is a bigger gap than any Apple Silicon upgrade can close. What breaks the cache: the Claude Code attribution header (fixed in v2.1.181, so update first), Ollama's default context window (32K on a 48GB Mac, smaller than Claude Code's ~33,000-token first request), hybrid models like Qwen3.8-27B that can't rewind past a checkpoint, frontends that send zero cached tokens, and tools that strip thinking tokens between turns. Most guides to speeding up a local LLM, whether that's switching Ollama to MLX, running a mixture of experts model or adding speculative decoding, make the writing faster. None of them stop an agent from re-reading its prompt from scratch. The chip still matters for cold reads. A 32,000-token first read on a 27B model takes minutes on an M1 Max and about 40 seconds on an M5 Max. So fix the cache first, then decide whether you need new local AI hardware. WHAT'S COVERED - Memory bandwidth explained: why it sets writing speed and not reading speed - Prefill vs decode, and why the LM Studio and Ollama counters only show decode - The same M5 Pro at 6.1 seconds and 268 seconds, and what changed between them - The Claude Code header bug and the CLAUDE_CODE_ATTRIBUTION_HEADER fix - Ollama's default context window vs Claude Code's first request (check it with `ollama ps`) - Why hybrid models lose the KV cache, and the llama.cpp log line that only shows at verbosity 4 - OpenClaw vs Hermes on the same server: zero cached tokens vs a working cache - Where the M5 Pro and M5 Max really do earn their price - A five-step checklist to run before you buy a faster Mac CHAPTERS 0:00 Eight hours, fifteen edits 1:17 The memory bandwidth case, stated fairly 3:08 Reading vs writing: prefill and decode 5:08 Same Mac, 6 seconds or 268 6:46 Cache killer 1: the Claude Code header 8:17 Cache killer 2: Ollama's 32K window 9:40 The model, the frontend, the thinking tokens 11:52 Where the chip does matter 13:14 The checklist SOURCES - 8 hours for 15 edits, M5 Max 128GB, LM Studio log: https://www.reddit.com/r/LocalLLM/comments/1u3xgtb/ - "MLX is not faster": 57 tok/s shown vs 3 tok/s effective, M1 Max: https://www.reddit.com/r/LocalLLaMA/comments/1rs059a/ - M5 Pro 64GB, Qwen3.8-27B, cache hit 6.1 s vs recompute 268.2 s: https://www.reddit.com/r/LocalLLaMA/comments/1wmbbf9/ - M5 Pro 48GB + Ollama + Claude Code, repeated compaction: https://www.reddit.com/r/LocalLLM/comments/1wgu6kn/ - OpenClaw 0 cached tokens, Hermes caches fine: https://www.reddit.com/r/LocalLLaMA/comments/1t9rlkc/ - M1 Max 32K first token, 672 s: https://www.reddit.com/r/LocalLLM/comments/1woq7cd/ - M5 Max 32K prefill, 615–827 tok/s: https://www.reddit.com/r/oMLX/comments/1wdjbbi/ - MacBook Pro M5 Max tech specs (614 GB/s): https://support.apple.com/en-us/126319 - Mac mini M6 tech specs (170 GB/s): https://support.apple.com/en-us/128108 - Apple, M5 Pro and M5 Max (Neural Accelerator in each GPU core): https://www.apple.com/newsroom/2026/03/apple-debuts-m5-pro-and-m5-max-to-supercharge-the-most-demanding-pro-workflows/ - llama.cpp Apple Silicon benchmarks: https://github.com/ggml-org/llama.cpp/discussions/4167 - Claude Code attribution block, v2.1.181: https://code.claude.com/docs/en/llm-gateway-protocol - Unsloth Claude Code guide: https://unsloth.ai/docs/basics/claude-code - Claude Code's first request, ~32,800 tokens (Systima via GIGAZINE): https://gigazine.net/gsc_news/en/20260713-claude-code-consuming-token - Ollama context length: https://docs.ollama.com/context-length - Ollama + Claude Code: https://docs.ollama.com/integrations/claude-code - Qwen3.8-27B model card: https://huggingface.co/Qwen/Qwen3.8-27B - Why hybrid models re-process the prompt: https://particula.tech/blog/prompt-reprocessing-swa-hybrid-models-kv-cache - JSyntax, "Local AI on a Mac Is a Waste of Money": https://www.youtube.com/watch?v=kLNDGwaHohE #LocalAI #LocalLLM #AppleSilicon