Supported models
| Family | Architecture | Status |
|---|---|---|
| Qwen3.8 (27B) and Qwen3.5 dense | hybrid attention + Gated DeltaNet | supported, with DFlash2 and MTP drafting |
| Qwen3.8-Flash-Next | mixture of experts, experts streamed from system RAM | supported (native build) |
| MiniCPM-V 4.7 35B-A3B (and other qwen35moe MoE models) | mixture of experts, text (vision input in development) | supported in 1.0.0, with DSpark drafting |
| GLM-5.3-Flash | mixture of experts, MLA attention | in development |
Release files
Qwen3.8-27B in four sizes, named by size. All are in
YTan2000/Qwen3.8-27B-TQ and need tqLLM.
Qwen3.8-27B-TQ3_4S-v2.gguf also runs in llama.cpp-tq3.
| File | Size | Good for |
|---|---|---|
Qwen3.8-27B-TQ_XS.gguf | 11.85 GB | 12-16 GB cards, the longest context |
Qwen3.8-27B-TQ_S.gguf | 12.55 GB | 16 GB cards, balanced |
Qwen3.8-27B-TQ_M.gguf | 13.14 GB | 16-24 GB cards (experimental: outside the quality-gated set, KLD only) |
Qwen3.8-27B-TQ_L.gguf | 14.72 GB | 24 GB cards, closest to the original |
Qwen3.8-27B-TQ3_4S-v2.gguf | 14.5 GB | tqLLM and llama.cpp-tq3 (YTan2000/Qwen3.8-27B-TQ3_4S) |
TQ3_4S, TQ3 and TQ4
All three rotate the weights before quantising. The rotation, the idea behind Google's TurboQuant, spreads outliers across a block so that a small codebook fits every block. They then store codes plus scales, and differ in block size, scales and where they run.
| Format | Block | Scales | Bits per weight | Runs in |
|---|---|---|---|---|
| TQ3_4S (GGUF type 46) | 32 weights, 32-point rotation, 8-level code | four 8-bit scales | 4.0 | tqLLM, llama.cpp-tq3 |
| TQ3 (type 301) | 128 weights, 128-point rotation, 8-level code | one FP16 per 128 | 3.125 | tqLLM |
| TQ4 (type 300) | 128 weights, 128-point rotation, 16-level code | one FP16 per 128 | 4.125 | tqLLM |
TQ3_4S spends a quarter of its bits on scales. TQ3 and TQ4 rotate over a block four times longer, so one scale per 128 weights is enough and the bits go into the codes instead. That is how the 27B fits in 11.85 GB as TQ3 against 14.5 GB as TQ3_4S-v2, and why TQ4 at about the same size as TQ3_4S is much closer to the original (KLD against BF16: 0.022 for the TQ4 file, TQ_L).
New to TQ3, TQ4 and TQ3_4S? TQ formats, explained shows how they work with pictures.
Supported tensor types
| Files that load | Q8_0, Q6_K, Q5_K_M / Q5_K_S, Q4_K_M / Q4_K_S, Q3_K_M / Q3_K_L, Q4_0, IQ4_XS, IQ4_NL, IQ3_S / IQ3_M, IQ3_XXS, Q2_K, IQ2_S / IQ2_XS / IQ2_XXS, IQ1_S (Unsloth UD quants included), F16, BF16, and our TQ3, TQ4 and TQ3_4S files |
|---|---|
| Tensor types | TQ3, TQ4, TQ3_4S, Q8_0, Q6_K, Q5_K, Q4_K, Q3_K, Q4_0, IQ4_XS, IQ4_NL, IQ3_S, IQ3_XXS, Q2_K, IQ2_S, IQ2_XS, IQ2_XXS, IQ1_S, F16, BF16 |
| Not yet | IQ1_M tensors |
| Architectures | Qwen3.5 / Qwen3.8 dense today; see Supported models. |
Drafters
Speculative decoding uses a small DFlash2 draft model: it proposes several tokens, the target model verifies them in one pass and commits the ones it would have produced itself. Greedy output is identical, token for token, to decoding without it.
| Drafter | Size | For |
|---|---|---|
drafters/Qwen3.8-27B-DFlash2-TQ3.gguf + mtp-vocab/atx_65536.txt | 0.79 GB | every Qwen3.8-27B file above |
The model's own MTP head (--mtp) | in the model | no extra file; one user, greedy |
Images
Pass a vision projector with --mmproj FILE to send images through the chat API. The projector
files (mmproj-F16.gguf, mmproj-BF16.gguf) are in
YTan2000/Qwen3.8-27B-TQ3_4S.
In development
- Mixture of experts on one consumer GPU. Experts that do not fit stay in system RAM; the hot ones live in VRAM, chosen from measured routing, with output identical to all-in-VRAM.
- More drafters. DSpark / DFlash drafters for MoE models, exact like DFlash2.
- More GPUs. A layer pipeline across several GPUs or machines.
Follow the repository for what lands next.