Essential cookies keep Instavar working. Optional analytics help us understand how the site is used and link your first-visit source to your Studio account after sign-in, for up to 180 days. Cookie Policy
Manage Cookie Preferences
Service reliability telemetry, including Sentry error monitoring and Vercel Speed Insights, stays enabled so we can secure the product and diagnose failures.
Breeze TTS 2 shipped as an inference model. We wanted to know whether its
synthesis stack could be adapted on one 24 GB GPU, which model components had
to change, and whether full supervised fine-tuning would improve on a much
smaller LoRA adapter.
We built the missing training surface, trained both routes on the consented
FEMALE_01 subset of Singapore's National Speech Corpus, and released the source
toolkit and selected research models.
60-second takeaway
Both training routes completed. The source toolkit reconstructs supervised
text and audio labels, trains LoRA or full SFT, resumes checkpoints, exports
artifacts and generates matched evaluation packs.
LoRA was the smaller practical release. The selected rank-8 adapter trains
12,092,448 parameters across the semantic backbone, depth decoder and three
projection families.
Full SFT changed much more of the synthesizer. The selected checkpoint
updates 2,387,151,872 synthesis parameters while keeping the text encoder and
audio codec frozen.
The objective comparison did not establish a winner. Full SFT reached mean
ECAPA speaker similarity 0.6973, compared with 0.6810 for LoRA. Both reached
0.0467 WER, and the paired ECAPA confidence interval crossed zero.
One blind listener preferred LoRA for cadence and long-form listening. The
three short-prompt comparisons were ties. Both models mispronounced the same
tested local word.
These are non-commercial research releases. The source code is Apache 2.0,
while the model artifacts remain governed by the BreezeBlue Research and
Non-Commercial License Agreement version 1.1.
The two Hugging Face repositories are public but gated. A user must accept the
licence conditions before downloading the files. The model cards remain visible
so that the provenance, intended use, evaluation and limitations can be reviewed
before access is requested.
What we had to add
The upstream release provided inference rather than a supported training
interface. A useful adaptation repository needed more than a loop that made the
loss decrease.
Our source toolkit adds:
deterministic reconstruction of text labels and 16-codebook target-audio
tokens;
a one-example BF16 forward-and-backward feasibility gate;
LoRA across the semantic backbone, depth decoder, text projection,
depth-input projection and output projection;
full SFT with a frozen text encoder and audio codec;
validation-based checkpoint selection and bounded hyperparameter sweeps;
matched generation, ASR, speaker similarity, acoustic diagnostics,
confidence intervals and opaque blind-listening packs; and
release checks that exclude training audio, optimizer state, caches and
private receipts.
The repository was exercised on an NVIDIA RTX 3090 Ti. This establishes the
documented paths on that 24 GB CUDA system. It does not show that every dataset,
GPU or sequence length will fit.
LoRA route
The selected LoRA artifact uses rank 8 and alpha 16. It came from step 500 of a
1,000-step schedule and contains 12,092,448 trainable parameters.
We did not restrict adaptation to the main transformer backbone. Breeze predicts
speech through both a semantic backbone and a depth decoder, with projections
connecting text, depth input and output tokens. The feasibility work showed that
all of those families participate in the supervised objective, so the released
adapter covers them together.
The adapter manifest pins the upstream model revision and required base-file
checksums. The custom loader refuses a mismatched revision or file identity. This
matters because an adapter can load successfully against the wrong mutable base
and still produce a misleading comparison.
Choose targets by testing the synthesis stack
Backbone-only LoRA was trainable, but that did not make it the best target set.
In a ten-step, one-example comparison at rank 8, adding the depth decoder
increased the total loss reduction from 0.008610 to 0.021196. Adding the text,
depth-input and output projections increased it to 0.023784 for another 82,976
trainable parameters. We held the example, base checkpoint, seed and optimizer
settings fixed, then saved, reloaded and merged each adapter.
That small comparison justified the next multi-example experiment, not a claim
of better speech. Finite gradients establish that a component can learn;
held-out generation and listening establish whether adapting it helps. The
custom non-Linear codebook head was outside this wrapper's targets, so this was
not an exhaustive target search.
Full-SFT route
The selected full-SFT artifact came from step 750 of a 1,000-step run. It updates
2,387,151,872 synthesis parameters while freezing the text encoder and audio
codec. The run used FP32-master SGD with BF16 model weights.
The public checkpoint contains inference roles only. It excludes optimizer,
scheduler, random state, trainer state, source recordings, caches and private
receipts. That smaller release boundary is intentional: users receive what is
needed for inference without receiving private data or unnecessary training
state.
Screen learning rates without changing the schedule underneath them
Our first full-SFT recipe completed mechanically but barely improved held-out
loss. Before another long run, we checked optimizer-state save and reload, then
screened learning rates on shorter trials.
The first 250-step screen had a confound: its cosine decay ended at step 250,
while the intended run ended at step 1,000. The same nominal learning rate did
not mean the same sequence of updates. We separated the trial stopping point
from the schedule horizon and repeated the finalists with a fixed 1,000-step
horizon. At step 250, SGD at 1e-2 beat 5e-3 on held-out loss for both tested
seeds. The longer run subsequently selected step 750 rather than step 1,000.
For a staged screen, preserve the learning rate at shared steps, warmup,
effective batch, data split and validation procedure. Stop the short trial
early without silently compressing the decay schedule. The selected rate is
specific to this optimizer and Breeze recipe; it is not a transferable AdamW
default. Loss selects candidates for audio evaluation, not a listening winner.
Test continuation against an uninterrupted control
A checkpoint loading successfully is weaker evidence than training continuing
from the intended state. In a separate two-update check, we compared an
uninterrupted run with a step-one pause and fresh-process continuation into a
new directory, binding the configuration, source revision and checkpoint files.
For full SFT, the final model, optimizer, scheduler and random-state hashes
matched. For LoRA, the adapter, scheduler and random-state files matched;
optimizer file bytes differed, but recursively compared tensor and scalar
values were equal. Process timing and first-step diagnostic metadata differed
as expected. Byte identity and equivalent loaded state answer different questions.
These checks used deliberate pauses, not arbitrary SIGTERM or power failures.
They qualify that small continuation case, not long-run reproducibility. Keep
atomic checkpoint completion, recovery after interruption and resumed-state
equivalence as separate tests.
Matched results
We compared the selected LoRA and full-SFT artifacts under the same
reference-free prompts and generation conditions.
Measurement
LoRA rank 8
Full SFT
Selected checkpoint
Step 500
Step 750
Mean ECAPA speaker similarity
0.6810
0.6973
Word error rate
0.0467
0.0467
Blind cadence preference
Preferred
Not preferred
Blind long-form preference
Preferred
Not preferred
The ECAPA difference is descriptive. Its paired confidence interval crossed
zero, so the checked objective sample did not establish that full SFT preserves
speaker identity better.
The listening study contained one listener. On neutral, ordinary Singapore
English and local-pronunciation prompts, the listener recorded ties with mixed
strengths. Full SFT sounded smoother in the neutral and local-pronunciation
pairs; LoRA sounded smoother in the ordinary Singapore English pair. Accent
impressions also varied by prompt. LoRA won both the cadence and long-form
comparisons, where it sounded more like ordinary-person narration and less
tiring. The recorded choices were two LoRA wins, three ties and no full-SFT wins.
This mixed result is useful. A larger checkpoint is not automatically the better
voice, and a speaker-embedding score does not measure cadence, pronunciation or
listening fatigue.
A tie is not necessarily sameness
The written notes mattered as much as the three-way choice. A listener could
prefer one sample's fluency and the other's accent, then select a tie. Counting
only wins would discard that distinction. A and B were randomized separately
for every pair, so we froze the ratings before joining exact sample IDs to the
private key. Display position was never a model identity.
Generic Singaporean accent and resemblance to this particular speaker also
need separate questions. Reference-free synthesis does not prevent an evaluator
from hearing a held-out target clip before rating identity. This first review
had no such anchor and used compressed Opus listening copies, so identity
impressions relied on memory. A stronger follow-up would use original WAVs,
a target anchor, separate fluency, identity, accent and fatigue ratings, and
additional listeners and prompts. More seeds alone do not supply more independent
prompts or listeners.
What the experiment does not prove
One voice profile does not establish performance across Singaporean English
speakers or accents.
One blind listener does not establish a population preference.
ECAPA similarity is a diagnostic proxy, not proof of identity fidelity.
WER checks whether an independent recognizer recovered the words. It does not
measure naturalness or pleasantness.
Both selected artifacts mispronounced the tested word paiseh. Neither route
solved local pronunciation merely by seeing Singapore English training data.
The releases are research artifacts, not evidence of production readiness or
commercial permission.
Which artifact should you try?
Start with LoRA when you want a small artifact, rapid iteration and an explicit
adaptation layer over a separately pinned base. In this experiment, LoRA also
received the stronger long-form listening preference.
Try full SFT when you specifically need to test whether deeper synthesis changes
improve your own voice and prompts, and you can carry a much larger checkpoint.
Our objective comparison did not prove that full SFT was better overall, so its
extra size should not be treated as quality evidence.
For either route, use held-out prompts and listen blind. Keep pronunciation,
accent, cadence, long-form monotony and fatigue separate from word accuracy and
speaker similarity.
Licence and responsible use
The GitHub toolkit is
source-only and licensed under Apache 2.0. It does not include model weights,
adapters, checkpoints, training audio or generated speech.
The upstream model and both Instavar derivatives are governed separately by the
BreezeBlue Research and Non-Commercial License Agreement version 1.1. Review the
licence and notices in each model repository before use. Do not use the released
models for a product, paid service, client work, advertising, revenue generation,
production deployment, impersonation, deception or non-consensual voice cloning.
The FEMALE_01 source material and our consent record do not override the model
licence. Dataset rights, voice consent, code licensing and model licensing are
separate checks.
Does an upstream update require retraining?
Not necessarily. In our checked Breeze v1.1 sync, the version label referred
to the agreement, not new synthesis weights. Comparing model objects across
the checked revisions found unchanged synthesis shards, codec, tokenizer and
configuration files. The source update repaired runtime behavior rather than
changing the supervised labels or loss. Both selected adaptations then completed
five compatibility prompts on the merged runtime without retraining.
Before repeating an expensive job, separate four questions: did the model
artifacts change, did the training objective change, did the runtime change,
or did only documentation and terms change? Compare pinned object identities,
not release names. If training inputs are unchanged, first test the existing
adapter or checkpoint in a fresh process. Reassess retraining when weights,
tokenization, codec, labels, loss or trainable targets change. This result applies
to that checked sync, not every later Breeze release; terms still require their
own review even when the weights are identical.
Listen to the reviewed examples
These paired clips retain the original audio, including the unsuccessful local
pronunciations. They illustrate the reviewed trade-offs rather than a general
model ranking.
Synthetic speech generated with adaptations of Breeze TTS 2 by BreezeBlue. These are non-commercial research examples, not a hosted synthesis service. Both variants use seed 42 without an inference-time reference recording. Original WAVs are unedited and not loudness-normalized.
Cadence
At first, nothing happened. Then the phone rang twice, the lift doors opened, and everyone turned around. No one spoke. After a moment, Priya laughed and the room relaxed.
On Saturday morning, I left home earlier than usual because the forecast warned of heavy rain. The station was already crowded, but the train arrived on time and the journey into town was quiet. I reviewed my notes, answered two messages, and watched the neighbourhoods pass outside the window. By the time I reached the office, the clouds had cleared. We spent the next hour checking the presentation, shortening a few crowded slides, and rehearsing the difficult transitions. The discussion with the client was calm and practical. They asked careful questions, we explained the trade-offs, and everyone agreed on the next steps. After lunch, I walked back through the park instead of taking the bus. It was a small change, but the slower route gave me time to think about what had gone well and what I would improve next time.
One open-label listener described LoRA as more natural in cadence and long-form listening, while finding full SFT clear but more monotonous. Both mispronounced Sze Min, Jalan Membina and paiseh. These judgments concern these clips, not all outputs or speakers. Playback levels differ.
The source repository documents installation, deterministic target preparation,
LoRA, full SFT, resume, merge and evaluation commands. It also records the exact
upstream revision required by the released artifacts.
Before trusting a result:
verify the base revision and checksums;
keep validation data separate from training data;
compare base, LoRA and full SFT with identical prompts and seeds;
check words, speaker similarity and acoustic measurements;
listen to an opaque pack without model identities; and
choose a checkpoint only after the objective and listening evidence agree, or
record the disagreement explicitly.