Local AI experiments,
with the receipts.
Practical experiments in local AI: quantization, speech, images, coding agents and the findings that changed what I run. Every result keeps its workload, source and limits.
- experiments
- 69
- supporting notebooks
- 86
- reviewed figures
- 56
- research areas
- 7
Latest from the lab
Recent experiments
A benchmark harness that keeps dry runs out of the results
The new harness layer provides four kinds of runs: streamed API throughput, GGUF quant comparison, task evaluation through lm-eval, and command-level memory/load measurement. It uses standard-library Python for the core, with optional external model and…
The archive now includes standard-library harnesses for throughput, quant comparison, task evaluation and VRAM measurement.
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
AWQ4 stayed close to the BF16 reference on WavLM and character errors; the sensitivity Q2 candidate was rejected.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
BF16 reduced median decoder time from 390.07 to 129.54 ms in Spanish and from 292.42 to 96.88 ms in English.
Keeping FP32 and moving Supertonic to Triton
Rather than reduce the CPU graph precision again, this follow-up used NVIDIA Triton’s ONNX backend with CUDA execution and the unchanged FP32 graphs. The paired comparison retained both speed and acoustic checks.
The paired GPU test cut median first-audio time to about 0.35 seconds while preserving the measured ASR results.
Longer Spanish phrases exposed the voice-transfer cost
Six phrases per language were synthesized with the production Supertonic stack, converted, then scored with an independent Parakeet recognizer. The experiment used public demo narrator references and did not alter the production application.
Speaker resemblance improved on longer phrases, but Spanish word error increased and conversion added over a second.
More diffusion steps did not reliably improve the voice match
The complete indexed collection contained fifty-one converted variants. Each was independently embedded against its exact selected narrator and source waveform. Twenty-nine used four steps, eleven eight and eleven twelve.
Matched six-sentence similarity intervals crossed zero for both eight-minus-four and twelve-minus-four steps.
Benchmarks with receipts
Every bar is a row in a file you can open.
Charts use the reviewed aggregate measurements, with their source revision, units and limits attached. Missing results stay unmeasured.
- Methods stay attached. Each report explains the workload, hardware and what remains untested.
- Measured trade-offs. Speed, memory and quality are reported separately; costs are included only when supported.
- Data you can inspect. Download aggregate CSV, JSON, SVG and PNG exports. Private source material stays private.
From A coding agent that knows when to ask for help
What recovery cost in three live scenarios
Single RTX 3090 · local Qwen worker + GLM reviewer · each scenario finished 10/10 checks
- Repeated failures · 2 reviews150
- Acceptance failure · 1 review142
- Successful task · no review9
seconds
Deliberately triggered integration scenarios; duration does not measure general coding accuracy.
View data & source
| Configuration | Value |
|---|---|
| Repeated failures · 2 reviews | 150 |
| Acceptance failure · 1 review | 142 |
| Successful task · no review | 9 |
Origin: structured measurements. Revision 6550ead3945b.
Lab Notes
Write-ups and deep dives
A benchmark harness that keeps dry runs out of the results
The new harness layer provides four kinds of runs: streamed API throughput, GGUF quant comparison, task evaluation through lm-eval, and command-level memory/load measurement. It uses standard-library Python for the core, with optional external model and…
MOSS on a 3090: Q2 failed, AWQ4 stayed closer
Every profile began with the pinned BF16 generator. The comparison reused fixed reference codes and an FP32 decoder for the main quality measurements, keeping codec changes separate from generator quantization.
Drama3: BF16 decoded the same codes about three times faster
The experiment replayed saved Spanish and English codes representing 21.5481 and 15.8824 seconds of audio. It compared FP32, BF16, mixed BF16 and two INT8 decoder variants on one RTX 3090.
Free Lab Notes
Follow the next experiment
Get new findings in your feed reader. All published articles and measurement exports are free to read.
Follow via RSSFollow the evidence
A notebook you can inspect
Read the methods, review the caveats, and explore the supporting notebooks behind the findings.
Explore the archive