TQ formats, explained

tqLLM stores model weights in its own TurboQuant formats: TQ3, TQ4 and TQ3_4S. Here is what they are, in plain words and pictures, and which file to download.

The short version

TQ33.125 bits per weight

The smallest. Fits a 27B model in about 12 GB, so it runs on 16 GB cards with long context.

TQ44.125 bits per weight

Closest to the original model. In our 27B files, more TQ4 cuts the drift from the original about 3× (KLD 0.079 → 0.022).

TQ3_4S4.0 bits per weight

The format that also runs in our llama.cpp fork. Small blocks with four scales each.

How it works, in four steps

A model is billions of numbers (weights). Storing each one in 16 bits is exact but huge. TQ formats store each in 3 or 4 bits instead, with as little damage as possible.

  1. Group. Take the weights a block at a time: 128 in a row for TQ3 and TQ4, 32 for TQ3_4S.
  2. Rotate. A few weights are much bigger than the rest (outliers). A fixed mathematical rotation mixes the block so the big values are shared out evenly. Nothing is lost: the engine rotates back when it computes.
  3. Snap. Each rotated weight is rounded to the nearest of a few levels: 8 levels (3 bits) or 16 levels (4 bits).
  4. Scale. One number per block (or per 8 weights in TQ3_4S) says how big the levels are.
Before rotation: one outlier forces every level to be coarse.
After rotation: values are even, so a few levels describe them all well.

That rotation is why one scale per 128 weights is enough. Plain 3-bit formats need much smaller blocks (more scales, more bits) to cope with outliers.

The three blocks

Each square is one weight; its shade is the level it was snapped to. The tall bars are the stored scales.

TQ3: 128 weights, 8 levels, one scale

128 × 3 bits + one 16-bit scale = 400 bits, or 3.125 bits per weight.

TQ4: 128 weights, 16 levels, one scale

128 × 4 bits + one 16-bit scale = 528 bits, or 4.125 bits per weight. Twice as many levels, so each weight lands much closer to its true value.

TQ3_4S: 32 weights, 8 levels, four scales

32 × 3 bits + four 8-bit scales (one per 8 weights) = 128 bits, or 4.0 bits per weight. Smaller blocks with their own scales; it is the format llama.cpp-tq3 can read.

The bit budget per weight

TQ33 code3.125
TQ3_4S3 code1 scale4.0
TQ44 code4.125

The original model uses 16 bits per weight, about four times more than any of these. TQ3_4S spends a full bit per weight on scales; TQ3 and TQ4 spend an eighth of one, because the rotation already tamed the outliers.

TQ3TQ4TQ3_4S
Weights per block12812832
Levels per weight8168
Scales per block1 (16-bit)1 (16-bit)4 (8-bit)
Bits per weight3.1254.1254.0
Runs intqLLMtqLLMtqLLM and llama.cpp-tq3

Which file to pick

Our release files mix TQ3 and TQ4 tensors: more TQ4 means closer to the original model and a bigger file. KL divergence measures how far a file's predictions drift from the 16-bit original (lower is closer).

FileSizeDrift from original (KLD)Same top tokenGood for
Qwen3.8-27B-TQ_XS11.85 GB0.07988.6 %16 GB cards, longest context
Qwen3.8-27B-TQ_S12.55 GB0.05590.1 %16 GB cards (default)
Qwen3.8-27B-TQ_M13.14 GB0.04591.2 %16-24 GB, experimental
Qwen3.8-27B-TQ_L14.72 GB0.02293.7 %24 GB cards, closest to the original

All four are in YTan2000/Qwen3.8-27B-TQ. The drafter never changes the answer, so the file only sets quality and memory, not correctness.

The rotation idea comes from Google Research's TurboQuant; see Credits.