The fastest kernel was not the qualified speech stack
The fitting setup combined HQ4, SDPA, the INT8 GPU codec, 0.8-second chunks, a 200-character budget and a 256-step cap. The BF16 comparison peaked at about 22.8 GiB, beyond the RTX 3060.
Key finding
HQ4, SDPA and the GPU INT8 codec formed the qualified fitting stack; a faster kernel experiment lacked complete audio evidence.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
Fitting within 12 GB does not establish BF16 voice parity. Generator-only and layer microbenchmarks do not qualify end-to-end speech.
Write-ups
Report moss-tts-hq4/rtx3060-best-stack-2026-10-08/README.md
A practical stack and a quality limit
The fitting setup combined HQ4, SDPA, the INT8 GPU codec, 0.8-second chunks, a 200-character budget and a 256-step cap. The BF16 comparison peaked at about 22.8 GiB, beyond the RTX 3060.
Voice control remained difficult: the instructed-pitch test passed only 1/21 HQ4 attempts against its recorded 195 Hz threshold. A BF16 English seed passed, while three Spanish cases failed. These are narrow control tests, not a general voice ranking.
Why I did not promote the fastest result
A Marlin experiment reached roughly 155 ms per generator step, or 6.46 tokens/s over a 48-token test with two runs. The production Triton path was around 180 ms. Complete waveform and token-tail evidence was lost, leaving the faster experiment unqualified for deployment.
A ConvRot W4A8 layer test showed roughly 1.74× speedup and cosine near 0.997, but ran under contention and did not render complete speech.
The decision
Keep complete output validation attached to performance. The October 8 service state was disabled/failed; this report does not claim that an experimental kernel was running in production.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.