About this app
**Model Overview**
Qwen3.8-27B is a vision-language model from the Qwen series (Alibaba), built on the Qwen3.5 architecture. It supports thinking, tool-calling, and native image/video understanding, with a 256K context window. This app serves the `UD-Q4_K_XL` GGUF build from `unsloth/Qwen3.8-27B-GGUF` via llama.cpp.
**Preset Configuration**
- Model: `unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL` (with `mmproj-F16.gguf` projector for vision)
- Capabilities: thinking, tools, vision
- Engine args: `-c 104448 -ngl all -fa on -ctk q8_0 -ctv q8_0 --jinja -np 2 -kvu --spec-type draft-mtp --spec-draft-n-max 2` (102K shared pool, 2 slots, MTP speculative decode)
- Required GPU memory: `23Gi`
**Accelerator modes**
| Mode | Device | Details |
|----------------|-------------------|--------------------------------------------------|
| `nvidia` | GPU (CUDA) | Uses `23Gi` GPU memory; `-ngl all` for full GPU |
| `nvidia-gb10` | GPU (Spark) | Like above, but memory is managed by the pod |
**How It Works**
First, Model Console downloads the model and gets it ready, then `llama.cpp` starts serving the model when it's ready. These two parts run separately but share files and progress info.
**Stack**
- **llama.cpp** (`server-cuda13-b11046`): Runs the model and serves API requests (port 8081)
- **Model Console**: Downloads the model, shows progress, and provides the OpenAI API on port 8090, proxying to `llamacpp`.
**Model Storage**
All model files are stored in a shared folder at `appCommon/huggingface`, mounted as `/cache/hf/hub`. All similar apps use this location for models.
What’s new
v1.0.18: Upgrade llama.cpp to server-cuda13-b11046. Raise the NVIDIA host-memory limit by 10Gi (engine 38Gi to 48Gi, chart limitedMemory 40Gi to 50Gi).
v1.0.17: Model Console (llm-init) updated to v1.7.21.
v1.0.16: Model Console upgrades to llm-init v1.7.19.
v1.0.14: Upgrade llama.cpp to server-cuda13-b10920. Keep the shared 102K context pool, two slots, and MTP draft depth 2.
v1.0.13: Upgrade llama.cpp to server-cuda12-b10884 and set the default MTP draft depth to 2. Migrate exact previous presets while preserving custom engine arguments.
v1.0.12: Budget host memory for the GPU memory that time-slicing evicts into RAM. The old limit was already fully consumed, which cost the container an OOM kill whenever another model took the card.
v1.0.11: Share the 102K KV pool across 2 slots (`-np 2 -kvu`) so one request can still use the whole window. Model Console moves to llm-init v1.7.14.
v1.0.5: Upgrade llm-init to v1.3.10; pin MODEL_REASONING_EFFORT and MODEL_REASONING_EFFORT_DEFAULT as non-editable env; upgrade llama.cpp to server-cuda12-b10548.
v1.0.4: Publish the reasoning levels this model actually accepts (low, medium, xhigh, default xhigh) on its model card, so a client can read them from Router instead of guessing. Requires llm-init v1.3.10.
v1.0.3: Lower the NVIDIA host-memory reservation so 1.12.6 timeslice nodes can install more easily (engine request 21Gi to 6Gi, chart requiredMemory 23Gi to 7Gi, covering the measured ~5.3Gi working set). GPU quota stays 23Gi.
v1.0.0: first release. llama.cpp server-cuda12-b10454, llm-init v1.3.5. Unsloth Qwen3.8-27B UD-Q4_K_XL + mmproj, 102K context (-c 104448), MTP --spec-draft-n-max 4, GPU quota 23Gi.