Making MOSS fit: the codec and the chunk budget
The HQ4 generator occupied about 6.24 GiB and the codec about 3.31 GiB. A one-shot path reached roughly 13 GiB and ran out of memory; 0.8-second streaming chunks kept the resident stack near 9.55 GiB on the 12 GB RTX 3060.
Key finding
GPU codec decoding cut several seconds from each tested render, but the token cap still required shorter text chunks.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
Native-voice comparisons and the changed female-reference encode are different cohorts. High waveform correlation does not qualify every cloned voice.
Data
Reviewed aggregate results with explicit units, sources and qualification limits. Different cohorts remain separate.
The codec shortened each tested native render
RTX 3060 · HQ4 · native-voice cases only
- 45 characters · before35.4
- 45 characters · GPU codec26.4
- 143 characters · before40.8
- 143 characters · GPU codec32.4
- 193 characters · before51.0
- 193 characters · GPU codec37.7
- Three sentences · before53.9
- Three sentences · GPU codec38.0
wall seconds
The female-reference encode changed separately. This chart does not establish cloned-voice parity.
View data & source
| Configuration | Value |
|---|---|
| 45 characters · before | 35.4 |
| 45 characters · GPU codec | 26.4 |
| 143 characters · before | 40.8 |
| 143 characters · GPU codec | 32.4 |
| 193 characters · before | 51.0 |
| 193 characters · GPU codec | 37.7 |
| Three sentences · before | 53.9 |
| Three sentences · GPU codec | 38.0 |
Origin: reviewed report table. Revision 7e6fdbe3b1d1.
Write-ups
Report moss-tts-hq4/rtx3060-gpu-codec-256cap-2026-10-06/README.md
Fitting the complete pipeline
The HQ4 generator occupied about 6.24 GiB and the codec about 3.31 GiB. A one-shot path reached roughly 13 GiB and ran out of memory; 0.8-second streaming chunks kept the resident stack near 9.55 GiB on the 12 GB RTX 3060.
Moving the final stage
For native-voice inputs of 45, 143 and 193 characters, wall time changed from 35.4/40.8/51.0 seconds to 26.4/32.4/37.7. The three-sentence case changed from 53.9 to 38.0. Codec decode fell from 10.3–18.6 seconds to 2.6–3.9.
Native waveform correlations ranged from about 0.9965 to 0.99993. The female-reference path also changed encoding and had a much lower correlation, so it cannot inherit the native-voice fidelity result.
The boundary that mattered
A 256-step ceiling needs a text budget. A 231-character fragment truncated at the ceiling; punctuation-aware chunks around a 200-character budget finished. FlashAttention2 was slower in the tested setup, around 241 ms per step versus 177 with SDPA, and changed the take. SDPA remained the qualified choice.
The useful outcome is a fitting pipeline with an explicit input limit. A smaller chunk is an acceptance rule, not just a speed preference.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.