2026-11-02 · base M4 24GB · also on x

41 tok/s from a 35B model on a base M4 mac mini with 24GB

my mac mini is the base M4 with 24GB. no pro chip, no egpu, nothing clever plugged into the back. it wrote a 1,024-token answer from a 35B mixture-of-experts model at 41.3 tok/s, the peak i have clocked. ten days earlier the same box with the same model did 22.

the hardware truth

base M4, 10 cores, 24GB unified memory. macOS only lets the gpu wire part of it. the cap is raised with sysctl iogpu.wired_limit_mb to 23040, which is 22.5 GiB. the model file is 14.44 GiB. after it there is room for one 16k context, a 0.22 GiB draft model, a 0.85 GiB vision projector and a 37 MB embedder. a 35B fits at all because only about 3B parameters fire per token.

the blueprint

llama.cpp built with metal. the model is MiniCPM-V-4.7-35B-A3B as a 3-bit mixed GGUF. the draft is a 237 MB Q4_K_M build made for the same model. the launch line, host flags removed:

llama-server -m ~/models/minicpm/m.gguf -c 16384 -ngl 99 -np 1 -fa on --jinja --metrics \
  -ctk q8_0 -ctv q8_0 --n-cpu-moe 0 --cache-ram 512 --prio 2 \
  --spec-type draft-dspark -md ~/models/minicpm/draft/dspark-MiniCPM-V-4_7-35B-A3B-Q4_K_M.gguf \
  -ngld 99 --spec-draft-n-max 4 --spec-draft-p-min 0.5 --spec-draft-sampling greedy

• -ngl 99 plus --n-cpu-moe 0 puts every layer and every expert on the gpu. that alone took it from 22.2 to 31.3.
• -c 16384 with q8_0 k and v cache is the biggest context that fits. -fa on is required for the quantized v cache.
• -np 1 is one slot. there is no memory for two.
• the draft flags: window of 4, keep proposing only while the draft is at least 50% sure, greedy picks. that took the median from 31.3 to 39.8.

benchmarks

a cold one-off benchmark of four prompts at 400 tokens each landed at 36.7, 40.1, 35.0 and 36.8, mean 37.2. so 41 is the peak on long answers, and a cold prompt lands closer to 37. a handful of answers in the logs hit 55 to 60. those stay out of the title.

draft settings that lost

window 15 ran at a median of 18.8. window 7 was 30.7. window 4 with p-min 0.8 was 32.9, with 0.7 it was 37.1. a big window guesses a long run, misses early, and the main model throws all of it away.

gotchas

• experts on the cpu were the worst trade. generation dropped to 22.2 and prefill to 106.7.
• a single slot means a long batch job can stall a chat. a priority proxy exists because of that.
• the box runs near the memory ceiling. a second model, a second slot or a 32k context will not fit.

more write-ups

• i finally priced the work running on my mini
• the draft looked fine until i checked the claim
• how paid frontier LLM burn dropped 91% with a 24GB M4 mac mini
• daily findings