Spanish Chatterbox: smaller weights, uneven speech

Start with the spoken output

The original BF16 baseline had recorded WER 0.42 and real-time factor 0.76 in this small Spanish screen. A dynamic, emulated FP8 path recorded WER 0.24 and an isolated RTF around 1.05. The user favored that candidate during listening, which is a personal observation rather than a blinded preference study.

The failures are part of the comparison

GPTQ reached RTF around 0.63–0.74 but produced silence tails, with WER 0.44. The GPTQ plus RPO variant reached WER 0.55. AWQ produced silence and WER 1.0. W4A8 recorded WER 0.37 but an RTF around 12, far too slow for the intended use.

Memory readings came from different co-resident states. They cannot serve as a uniform peak-memory leaderboard, and this screen does not establish an architectural limit for the GPU.

What I learned

A quantization label is not an acceptance test. Keep intelligibility, silence detection, end-of-speech behavior and listening checks alongside memory and timing. The next comparison needs more speakers and text, fixed residency, matched timings and blinded listening before choosing a default.

Evidence and publication boundary

This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.

• Source record 1