Credits
tqLLM re-implements ideas from the work below. No code is copied unless it is listed in the repository's NOTICE file. Thank you to every author.
| Piece | Credit |
|---|---|
| Qwen3.8-27B weights, MTP head, chat template | Qwen team, Alibaba (Qwen) |
| GGUF format, K-quant / IQ block layouts, reference engine; base of our llama.cpp-tq3 fork | ggml authors, llama.cpp, Georgi Gerganov |
| DFlash / DFlash2 drafting method and drafter weights (served here in TQ3 / TQ4 with a 65,536-token draft vocabulary) | Jian Chen, Yesheng Liang, Zhijian Liu, z-lab (arXiv 2602.06036) |
| Rotation-based quantisation, the lineage of the TQ3 / TQ4 weight formats and the tq3 KV cache | Google Research, TurboQuant |
| turbo2 / turbo3 / turbo4 KV types, REFRACT quality metric | Tom Turney (TheTom), turboquant_plus (X: @no_stp_on_snek) |
| Trellis-coded KV idea (tcq codecs; codebooks and kernels ours) | spiritbuun (buun), buun-llama-cpp (X: @spiritbuun) |
| Learned rotations for quantisation | Avarok, TorQuant |
| Fixed chat template (v22, embedded in our files) | froggeric, Qwen-Fixed-Chat-Templates |
| FlashAttention tiling | Tri Dao et al., flash-attention |
| Speculative sampling | Leviathan, Kalman, Matias (arXiv 2211.17192); Chen et al. (arXiv 2302.01318) |
| Token-tree verification (our hybrid Gated DeltaNet tree kernels build on it) | SpecInfer, Miao et al. (arXiv 2305.09781); Medusa, Cai et al.; EAGLE, Li et al. |
Prompt-lookup drafting (--lookup) | Apoorv Saxena, prompt-lookup-decoding |
| Comparison baselines | Unsloth UD quants; HyperQwen; ExLlamaV3, turboderp; vLLM; Strata, Niko1221; LocalMaxxing |
How tqLLM is built
tqLLM is built by one person and a small team of AI agents that coordinate on Agora, a shared task board. Every change goes through the same loop: a task, a plan, the work, a test on real GPUs, a review, and a recorded result.
- Owner sets goals and says go.
- Agora, the shared task board: tasks, plans, reviews and every result with its GPU, files and settings.
- AI lead plans, reviews and assigns work.
- AI tester gates every change on real GPUs: exact output, speed and quality.
- AI research reads the research catalog, which Hermes updates daily, and proposes ideas.
- Alpha and Omega, self-improvement loops running QwenGyre on our own GPUs, send patches and measurements back.
Testers
Thank you to the testers from X who ran tqLLM on their own hardware and reported back: 0x443, Ricky Spanish, lackonendes and GandalfOne.
tqLLM itself is licensed under Apache-2.0 (LICENSE and NOTICE in the repository). Model weights and bundled third-party source keep their own licenses.