Appearance
Compare Qwen3.8-27B token generation and prompt processing speeds across tested local AI hardware, with single-request and four-request results.
20 prompts, medium reasoning
For configurations marked queued, the server runs one request at a time. Four clients wait on that single stream.
Prefill speed reported by the engine, tok/s, median of 3 runs per prompt length
Prompt length in tokens. Speed in tokens per second.
Scroll horizontally to compare all prompt lengths.
| Machine | Configuration | 1k | 4k | 16k | 32k | 64k |
|---|---|---|---|---|---|---|
| Olares Onellama.cpp, NVFP4 LOW ↗ | llama.cpp, NVFP4 LOW ↗ | 2,005 | 2,713 | 2,716 | 2,255 | 1,510 |
| Olares Onellama.cpp, UD-Q4_K_XL ↗ | llama.cpp, UD-Q4_K_XL ↗ | 1,545 | 1,671 | 1,602 | 1,434 | 1,114 |
| DGX Sparkllama.cpp, NVFP4 LOW ↗ | llama.cpp, NVFP4 LOW ↗ | 950 | 1,072 | 1,061 | 1,007 | 877 |
| Mac Studio M5 MaxOllama (MLX), nvfp4 ↗ | Ollama (MLX), nvfp4 ↗ | 975 | 1,016 | 951 | 874 | 737 |
| Mac Studio M5 MaxOllama (GGML), UD-Q4_K_XL ↗ | Ollama (GGML), UD-Q4_K_XL ↗ | 690 | 712 | 677 | 594 | 497 |
| GTR9 Prollama.cpp Vulkan, UD-Q4_K_XL ↗ | llama.cpp Vulkan, UD-Q4_K_XL ↗ | 318 | 326 | 300 | 268 | 223 |