Appearance
Compare Qwen3.8-Flash-Next inference speed on Olares One, NVIDIA DGX Spark and Mac Studio M5 Max. Review IFBench scores for three quantizations tested on Olares One.
We tested five quantizations with Strata 0.2.10. Coverage varies by device and task.
IFBench is an instruction-following benchmark from Ai2. It tests whether models follow explicit requirements in their responses.
300 prompts, xhigh reasoning, 65,536-token output limit
Each configuration was evaluated once on Olares One. Scores may vary between runs. IQ3_S and Q2_0 were tested for speed only. Truncated outputs reached the 65,536-token limit.
Evaluation time
Lower is better| Machine | Configuration | Score | Truncated | Output per question | Token generation | Evaluation time |
|---|---|---|---|---|---|---|
| OfficialModel card ↗ | Original weights, model card ↗ | 81.3 | — | — | — | — |
| Olares OneStrata, IQ2_XS ↗ | Strata, IQ2_XS ↗ | 76.3 | 9 | 13,669 tok | 148 tok/s | 7.7 h |
| Olares OneStrata, IQ3_XXS ↗ | Strata, IQ3_XXS ↗ | 82.0 | 1 | 11,827 tok | 124 tok/s | 8.0 h |
| Olares OneStrata, UD-IQ4_XS ↗ | Strata, UD-IQ4_XS ↗ | 84.0 | 1 | 9,334 tok | 88 tok/s | 8.8 h |
One run varies by about ±2.3 points at this question count, so a score above the official one is noise, not a gain. Evaluation time is about questions × output per question ÷ token generation speed, one request at a time. It also includes loading and scoring.
Prompt processing and single-request token generation across five quantizations tested with Strata.
20 prompts, medium reasoning, 3,072-token output limit
One request at a time. Hardware and Strata settings differ between systems. Q2_0 has speed results only, with no IFBench evaluation.
Token generation, single request
Higher is better| Machine | Configuration | Token generation | Time to first token |
|---|---|---|---|
| Olares OneStrata, Q2_0 ↗ | Strata, Q2_0 ↗ | 166 tok/s | 0.29 s |
| Olares OneStrata, IQ2_XS ↗ | Strata, IQ2_XS ↗ | 162 tok/s | 0.32 s |
| Olares OneStrata, IQ3_XXS ↗ | Strata, IQ3_XXS ↗ | 133 tok/s | 0.34 s |
| Mac Studio M5 MaxStrata, Q2_0 ↗ | Strata, Q2_0 ↗ | 130 tok/s | 0.25 s |
| Olares OneStrata, IQ3_S ↗ | Strata, IQ3_S ↗ | 113 tok/s | 0.39 s |
| Olares OneStrata, UD-IQ4_XS ↗ | Strata, UD-IQ4_XS ↗ | 92 tok/s | 0.43 s |
| DGX SparkStrata, Q2_0 ↗ | Strata, Q2_0 ↗ | 80 tok/s | 0.53 s |
| DGX SparkStrata, IQ3_XXS ↗ | Strata, IQ3_XXS ↗ | 68 tok/s | 0.60 s |
| DGX SparkStrata, IQ3_S ↗ | Strata, IQ3_S ↗ | 59 tok/s | 0.64 s |
| DGX SparkStrata, UD-IQ4_XS ↗ | Strata, UD-IQ4_XS ↗ | 52 tok/s | 0.64 s |
| Mac Studio M5 MaxStrata, IQ3_XXS, no MTP ↗ | Strata, IQ3_XXS, no MTP ↗ | 47 tok/s | 1.01 s |
| Mac Studio M5 MaxStrata, IQ3_S, no MTP ↗ | Strata, IQ3_S, no MTP ↗ | 41 tok/s | 1.35 s |
Client-observed prompt processing speed, median of three runs per prompt length
Calculated as prompt tokens divided by time to first token. This differs from the engine-reported metric on the Qwen3.8-27B page. Use the same metric when comparing models.
Prompt length in tokens. Speed in tokens per second.
| Machine | Configuration | 1k | 4k | 16k | 32k | 64k |
|---|---|---|---|---|---|---|
| Olares OneStrata, Q2_0 ↗ | Strata, Q2_0 ↗ | 1,257 | 2,536 | 3,261 | 3,321 | 3,312 |
| Olares OneStrata, IQ3_S ↗ | Strata, IQ3_S ↗ | 910 | 2,235 | 2,522 | 2,603 | 2,447 |
| Olares OneStrata, IQ3_XXS ↗ | Strata, IQ3_XXS ↗ | 1,034 | 2,349 | 2,374 | 2,285 | 2,359 |
| Olares OneStrata, IQ2_XS ↗ | Strata, IQ2_XS ↗ | 1,181 | 1,834 | 2,268 | 2,295 | 2,285 |
| Mac Studio M5 MaxStrata, Q2_0 ↗ | Strata, Q2_0 ↗ | 1,998 | 2,066 | 2,128 | 2,105 | 2,074 |
| DGX SparkStrata, UD-IQ4_XS ↗ | Strata, UD-IQ4_XS ↗ | 667 | 1,667 | 2,095 | 2,183 | 2,169 |
| DGX SparkStrata, IQ3_XXS ↗ | Strata, IQ3_XXS ↗ | 694 | 1,693 | 2,006 | 1,885 | 2,058 |
| DGX SparkStrata, Q2_0 ↗ | Strata, Q2_0 ↗ | 766 | 1,477 | 1,892 | 1,880 | 1,893 |
| Olares OneStrata, UD-IQ4_XS ↗ | Strata, UD-IQ4_XS ↗ | 782 | 1,219 | 1,872 | 1,850 | 1,867 |
| DGX SparkStrata, IQ3_S ↗ | Strata, IQ3_S ↗ | 620 | 1,554 | 1,865 | 1,933 | 1,966 |
| Mac Studio M5 MaxStrata, IQ3_XXS, no MTP ↗ | Strata, IQ3_XXS, no MTP ↗ | 565 | 613 | 624 | 616 | 590 |
| Mac Studio M5 MaxStrata, IQ3_S, no MTP ↗ | Strata, IQ3_S, no MTP ↗ | 431 | 455 | 463 | 463 | 449 |
All tests used Strata 0.2.10 with engine 0.1.56 and one request at a time. Quantization and runtime settings affect both speed and quality.
IQ2_XS failed to load on DGX Spark in this test and is not offered by Strata on macOS. UD-IQ4_XS was not tested on the Mac. These missing results do not represent zero performance.
For comparison, see Qwen3.8-27B benchmarks. Its IFBench results on Olares One average three runs for the NVFP4 LOW configuration. Flash-Next scores here are single runs and do not establish a consistent quality advantage.