daily findings

what the box measured

read-only pulls from the llama-server timing log and /metrics. times are CT.

2026-11-09

collected 10:05 AM CT · Nov 9, 2026

median decode since midnight: 51.5 tok/s across 688 answers of 100+ tokens on the base M4 24GB (fastest 67.7 tok/s, 74% of answers at 40 tok/s or more)
speculative draft acceptance since midnight: 86.6% (916,246 of 1,057,874 drafted tokens kept)
median prompt prefill since midnight: 376.5 tok/s on prompts of 500+ tokens (693 prompts)
tokens handled on the local model since midnight: 8,209,911 (5,059,083 prompt, 3,150,828 generated), none of them billed by a cloud API
server-wide decode rate since midnight: 53.2 tok/s (3,150,828 tokens in 986.6 minutes of generation)
the prompt cache had to evict its oldest entry 2656 times since the last server start to make room
swap in use at 10:05 am: 3,842 MB of 5,120 MB

collected 11:09 PM CT · Nov 8, 2026

median decode since midnight: 44.9 tok/s across 657 answers of 100+ tokens on the base M4 24GB (fastest 68.4 tok/s, 95% of answers at 40 tok/s or more)
speculative draft acceptance since midnight: 87.1% (1,262,986 of 1,449,826 drafted tokens kept)
median prompt prefill since midnight: 373.2 tok/s on prompts of 500+ tokens (262 prompts)
tokens handled on the local model since midnight: 4,148,549 (1,602,997 prompt, 2,545,552 generated), none of them billed by a cloud API
server-wide decode rate since midnight: 44.8 tok/s (2,545,552 tokens in 947.8 minutes of generation)

collected 11:03 PM CT · Nov 7, 2026

median decode since midnight: 43.7 tok/s across 2086 answers of 100+ tokens on the base M4 24GB (fastest 52.0 tok/s, 77% of answers at 40 tok/s or more)
speculative draft acceptance since midnight: 86.6% (3,799,037 of 4,386,936 drafted tokens kept)
median prompt prefill since midnight: 346.9 tok/s on prompts of 500+ tokens (2064 prompts)
tokens handled on the local model since midnight: 4,384,480 (1,609,180 prompt, 2,775,300 generated), none of them billed by a cloud API
server-wide decode rate since midnight: 43.1 tok/s (2,775,300 tokens in 1073.5 minutes of generation)

collected 11:02 PM CT · Nov 6, 2026

median decode since midnight: 43.2 tok/s across 1816 answers of 100+ tokens on the base M4 24GB (fastest 54.9 tok/s, 71% of answers at 40 tok/s or more)
speculative draft acceptance since midnight: 86.4% (2,744,869 of 3,178,318 drafted tokens kept)
median prompt prefill since midnight: 356.4 tok/s on prompts of 500+ tokens (2290 prompts)
tokens handled on the local model since midnight: 9,318,450 (6,576,380 prompt, 2,742,070 generated), none of them billed by a cloud API
server-wide decode rate since midnight: 42.9 tok/s (2,742,070 tokens in 1065.6 minutes of generation)
the prompt cache had to evict its oldest entry 2451 times since the last server start to make room
swap in use at 11:02 pm: 2,397 MB of 3,072 MB

collected 11:09 PM CT · Nov 5, 2026

median decode since midnight: 43.9 tok/s across 1065 answers of 100+ tokens on the base M4 24GB (fastest 52.6 tok/s, 73% of answers at 40 tok/s or more)
speculative draft acceptance since midnight: 86.6% (1,812,655 of 2,093,344 drafted tokens kept)
median prompt prefill since midnight: 355.4 tok/s on prompts of 500+ tokens (1563 prompts)
tokens handled on the local model since midnight: 9,064,250 (6,090,230 prompt, 2,974,020 generated), none of them billed by a cloud API
server-wide decode rate since midnight: 43.8 tok/s (2,974,020 tokens in 1132.2 minutes of generation)
the prompt cache had to evict its oldest entry 3826 times since the last server start to make room
swap in use at 11:09 pm: 5,147 MB of 6,144 MB

collected 11:15 PM CT · Nov 4, 2026

median decode since midnight: 44.1 tok/s across 1239 answers of 100+ tokens on the base M4 24GB (fastest 54.9 tok/s, 82% of answers at 40 tok/s or more)
speculative draft acceptance since midnight: 87.1% (2,079,974 of 2,387,725 drafted tokens kept)
median prompt prefill since midnight: 348.6 tok/s on prompts of 500+ tokens (1026 prompts)
tokens handled on the local model since midnight: 5,775,749 (2,483,724 prompt, 3,292,025 generated), none of them billed by a cloud API
server-wide decode rate since midnight: 44.1 tok/s (3,292,025 tokens in 1245.2 minutes of generation)
the prompt cache had to evict its oldest entry 2783 times since the last server start to make room
swap in use at 11:15 pm: 5,304 MB of 6,144 MB

median decode since midnight: 39.8 tok/s across 260 answers of 100+ tokens (fastest 58.8, 47% of answers at 40 or more)
speculative draft acceptance: 76.2% (78,739 of 103,292 drafted tokens kept)
median prompt prefill: 360.8 tok/s on 311 prompts of 500+ tokens
tokens handled locally by 12:02 pm CT: 758,118 (606,090 prompt, 152,028 generated), none billed by a cloud api
the 8:00 hour was the fastest at a 46.6 tok/s median across 42 answers
swap in use at 12:02 pm: 5,670 MB of 6,144 MB. the prompt cache evicted its oldest entry 200 times

median decode from the 8:28 am server start: 38.6 tok/s across 618 answers (fastest 55.1)
speculative draft acceptance: 76.2% (217,206 of 285,211 drafted tokens kept)
median prompt prefill: 358.9 tok/s on 742 prompts of 500+ tokens
tokens handled locally: 1,964,784 (1,612,739 prompt, 352,045 generated), none billed
server-wide decode rate: 37.5 tok/s, 352,045 tokens in 156.6 minutes of generation
the prompt cache evicted its oldest entry 413 times to make room

the live config throughout: draft window 4, p-min 0.5, greedy draft sampling, gpu wired cap 22.5 GiB of 24.