Releases

What changed in each version. Docker images: ytan2000/tqllm:<version> and ghcr.io/turbo-tan/tqllm:<version> (x86-64 and arm64 in one tag); latest is the newest. The full list is the CHANGELOG.

1.2.3 · 22 Oct 2026

Qwen3.8 Flash-Next at full speed from the Hugging Face files, no pack file. RTX 3090 from a cold start: chats 96-103 tok/s and a 32K prompt's first token in 14.6 s (1.2.1 / 1.2.2: 79-88 tok/s and 17.3 s); output identical.

  • The gap was the per-layer token embedding (28.8 GB, read row by row at every token) not being cached, so new tokens waited on the disk. The engine now reads it once at start-up (about 6 s).
  • Correction to 1.2.1: the optional 50.9 GB pack file does not change speed and is no longer needed.
  • Further compression is on the way.

1.2.2 · 22 Oct 2026

Qwen3.8 Flash-Next on GB10 / DGX Spark from the Hugging Face files. With 1.2.1 the published manifest failed on a GB10 (its expert cache settings were an RTX 3090's). The engine now sizes the expert cache, its memory cap and the prefill chunk on the device at start-up; the manifest only lists files.

  • For about 30 minutes on 22 October the Hugging Face manifest lacked a line that 1.2.1 needs, which made long prompts slow (a 32K prompt took minutes). It was restored, and 1.2.2 no longer depends on it.
  • Every release now runs the Hugging Face card's commands verbatim on an RTX 3090 and a GB10 before it ships.
  • RTX 3090: output identical to 1.2.1, same speed.

1.2.1 · 22 Oct 2026

Qwen3.8 Flash-Next runs from the public files alone. The 51 GB packed-expert file is no longer needed: at start-up tqLLM arranges the experts of the ISTA-DASLab IQ3_S GGUF in memory itself (about 50 s; the server is ready in about two minutes), with bytes identical to the file. Setup: YTan2000/Qwen3.8-Flash-Next-tqLLM.

  • Short chats decode about 15 % slower this way on an RTX 3090 (79-87 against 93-102 tok/s), and a 32K prompt takes 17.3 s to its first token instead of 14.6 s. Long-context decode is the same. (Fixed in 1.2.3: the cause was an uncached embedding table, not the pack file.)
  • Coming next: a single, more compact Flash-Next file with the experts already arranged: no side files and full speed from the start.
  • The engine drops the GGUF's page cache after building, and the manifest carries the per-layer expert cache plan (layer_slots=).

1.2.0 · 22 Oct 2026

Qwen3.8 Flash-Next (125B MoE) on one RTX 3090. Against Strata 0.2.2 on the same GGUF and the same card: 1.63x faster decode after a 128K-token prompt (77.1 against 47.2 tok/s), 1.34x after 64K, about 3x sooner first token on short chats (0.32-0.41 s against 1.07-1.29 s), chats level or ahead (91.6 against 89.6 tok/s). Method and charts.

  • CUDA graphs for Flash-Next decode on PCIe GPUs and expert slot tiers: expert banks sized to the free memory and shared across layers, so long prompts keep decoding fast.
  • Adaptive draft length for native MTP on larger cards (up to 5 drafts), output identical to fixed drafting.
  • Native MTP drafting on mixture-of-experts files with an MTP block (--mtp): Qwen3.6-35B-A3B on an RTX 3090 246-330 tok/s over five chats (178-183 without MTP), output identical.
  • System-RAM experts for files larger than the GPU (--moe-host-mb auto) with a VRAM cache of host experts (--moe-cache-mb).
  • Built-in monitor on the API port: live prefill progress and throughput, request telemetry and a terminal chat.
  • OpenAI Responses API (/v1/responses, streaming and tool calls): Codex CLI connects directly.
  • tqllm launch pi and tqllm launch dsh; default sampling for agents (--temperature / --top-p / --top-k).
  • Fixes: small cards (12 GB) decode and long-prompt refill, GB10 long prompts that failed a few MB short of the prefill plan, 262K-context requests during CUDA graph capture, the Docker build.

docker pull ytan2000/tqllm:1.2.0

1.0.0 · 14 Oct 2026

First public release. Qwen3.8-27B in the TQ release files (TQ_XS 11.85 GB to TQ_L 14.72 GB) on one GPU, with exact speculative decoding.

  • Qwen3.8 / Qwen3.5 dense models and qwen35moe mixture-of-experts files (e.g. MiniCPM-V 4.7 35B-A3B); TQ3, TQ4, TQ3_4S and the common GGUF K-quant / IQ types.
  • DFlash2 drafter with tree and chain verify, the model's own MTP head, DSpark drafters: greedy output identical to plain decoding in every mode.
  • Automatic memory fit, compressed KV caches (tq3, turbo2/3/4, tcq2/3, q4), TqCache for context beyond the GPU, up to 256K context on one RTX 3090.
  • Qwen3.8-27B on one RTX 3090: 273.1 tok/s on LocalMaxxing reasoning-v1 (approved), 199.5 on code-v1.
  • OpenAI-compatible server with tool calls and reasoning controls, --np parallel slots, tqllm launch opencode | claude, Docker image for x86-64 and arm64.

docker pull ytan2000/tqllm:1.0.0