Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
Key finding
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.
Edited finding from the pinned experiment record. The report and figure metadata preserve the original workload and limits.
The boundary
The 96-render comparison and later 48-request production validation are separate cohorts. Server timings exclude a complete physical two-iPhone call.
Data
Reviewed aggregate results with explicit units, sources and qualification limits. Different cohorts remain separate.
FP32 GPU execution reduced first-audio delay
96 renders · 12 phrases per language · two repeats · paired CPU/GPU
- English · CPU1.154
- English · Triton GPU0.349
- Spanish · CPU1.079
- Spanish · Triton GPU0.341
median seconds to first PCM byte
NVIDIA Triton ONNX backend with CUDA execution. Excludes ASR, phone capture and call network delay.
View data & source
| Configuration | Value |
|---|---|
| English · CPU | 1.154 |
| English · Triton GPU | 0.349 |
| Spanish · CPU | 1.079 |
| Spanish · Triton GPU | 0.341 |
Origin: reviewed report table. Revision 7e6fdbe3b1d1.
Write-ups
Report tts/supertonic3-precision-triton-2026-10-13/triton/REPORT.md
A different optimization
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
Faster first audio
English median TTFT changed from 1.154 seconds on CPU to 0.349 on the GPU route. Spanish changed from 1.079 to 0.341. Median RTF changed from 0.353 to 0.109 and from 0.345 to 0.107. Measured WER matched within each language, and waveform correlation exceeded 0.9994. The route used about 989 MiB of GPU memory.
Check the deployed path separately
A later forty-eight-request production API check passed. English TTFT was about 341 ms with RTF 0.110 and WER 3.57%; Spanish about 340 ms, RTF 0.107 and WER 3.09%. These numbers should not be pooled with the earlier paired corpus.
Synthetic two-way server exercises combining ASR, translation and synthesis passed with about 0.92/0.82 seconds of server turnaround. The physical phone was installed but locked, so this does not prove microphone capture, international network delay or real two-phone conversation acceptance.
Evidence and publication boundary
This is an edited account of the recorded experiment. The linked archive is private; the methods, aggregate findings and limits are included above. Reference recordings, raw conversations, authentication material and workstation configuration are excluded.
More in Speech & codecs
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Longer Spanish phrases exposed the voice-transfer cost
Six phrases per language were synthesized with the production Supertonic stack, converted, then scored with an independent Parakeet recognizer. The experiment used public demo narrator references and did not alter the production application.
Speaker resemblance improved on longer phrases, but Spanish word error increased and conversion added over a second.