Technical archive / model selection

Five TTS engines, with the trade-offs left visible.

Selection considers similarity, expression controls, long-form stability, memory, sample rate, integration cost, and model-weight licensing. These are the five paths implemented in the workstation, documented for clients and machine retrieval.

5engine paths
3voice modes
22.05-44.1 kHzoutput range
100k+single batch

Decision table

Select by deliverable, then tune the parameters.

GoalFirst candidateStrengthTrade-off
Stable Chinese long formIndexTTS 2.5Default MLX 8-bit path with cached conditioningNo native eight-emotion vector in this adapter
Explicit emotion categoryIndexTTS 2.0Emotion type and strength are separateOlder engine; benchmark on target hardware
Text-described voiceOmniVoice MLXClone, design, and automatic modesCloning depends on an aligned transcript
Multiple roles and tagsFish Audio S2 ProSpeaker mapping, inline expression, 44.1 kHzHigher memory and licensing requirements
Native engine validationVoiceStudioPyTorch/MPS reference pathHeavier runtime and subprocess coordination

Engine 01

IndexTTS 2.5 / MLX 8-bit

The default path accepts target text and reference audio, creates model-specific voice conditioning, caches it, and emits 22,050 Hz audio.

Contracttext + reference_audio → condition → waveformWAV / MP3 / FLAC

Strengths

  • Efficient fit for Apple Silicon unified memory.
  • Short, reliable path for Chinese narration.
  • Conditioning cache helps recurring voices.
  • Integrated chunking, progress, cancellation, and partial results.

Limitations

  • No native IndexTTS 2.0 eight-emotion vector in this adapter.
  • Long form still requires chunks; project scale is not one unbounded inference.
  • Caches are model-version specific.
  • Extreme inputs can require shorter guarded retries.

Engine 02

IndexTTS 2.0 / MLX

The explicit emotion route combines text and a reference voice with one of eight categories plus an independent strength value.

Contracttext + audio + emotion + strength → waveformhappy, sad, angry, afraid, disgusted, melancholic, surprised, calm

Strengths

  • Structured and repeatable emotion parameters.
  • Useful for ads, drama, and emphasized narration.
  • Shares the workstation voice library and export pipeline.

Limitations

  • Quality and speed must be sampled rather than inferred from version numbers.
  • High emotion strength may reduce naturalness or similarity.
  • Chunk joins need consistent loudness and pauses.

Engine 03

OmniVoice / MLX

One backend supports cloning, text-described voice design, and automatic voice. Clone mode pairs reference audio with a word-aligned transcript; local ASR can produce a draft when absent.

Contractsclone: text + audio + transcriptdesign: text + voice_description24,000 Hz

Strengths

  • Three voice modes in one engine.
  • Voice design works without reference audio.
  • Expression presets include whisper-like delivery.
  • Local ASR reduces missing-input friction.

Limitations

  • Transcript mistakes directly affect clone conditioning.
  • Natural-language styles are less deterministic than numeric vectors.
  • ASR adds latency and recognition risk.
  • Upstream weight terms require use-case review.

Engine 04

Fish Audio S2 Pro / MLX 8-bit

The expressive multi-speaker route caches reference encodings, splits long scripts at punctuation into roughly 60-character safe units, and merges 44,100 Hz output.

Contracttext/tags + reference + speaker_map → chunks → waveform44,100 Hz

Strengths

  • Multi-speaker, automatic, and reference-clone modes.
  • Inline expression tags and speaker routing.
  • Higher output sample rate for downstream production.
  • Cached references suit episodic work.

Limitations

  • Greater memory pressure and device-dependent speed.
  • Long unbroken inputs risk token truncation.
  • Role syntax and names must remain consistent.
  • Commercial use requires the appropriate Fish Research License.

Engine 05

VoiceStudio native OmniVoice / PyTorch MPS

The native compatibility path runs in an isolated subprocess. The workstation owns inputs, state, and outputs while the subprocess owns the heavier model lifecycle.

Contractjob.json → subprocess → progress/result.json → audioclone / design / auto

Strengths

  • Reference path for comparing the MLX adaptation.
  • Dependency and failure isolation.
  • All three OmniVoice modes retained.
  • Subprocess failure does not erase the main UI state.

Limitations

  • Heavier startup and memory footprint.
  • The audio tokenizer may run on CPU.
  • Progress, cancellation, and cleanup cross a process boundary.
  • Upstream licensing still requires review.

Long-form scale

100,000 or 200,000 characters is a job-orchestration problem, not one unlimited inference.

  1. Create stable IDs for books, chapters, sections, and batches.
  2. Split each batch at punctuation within the safe length of the selected model.
  3. Record text hash, engine, voice, parameters, output path, and status per segment.
  4. Retry only failed segments and resume from checkpoints.
  5. Normalize sample rate, loudness, silence, and naming before chapter or full-project assembly.
  6. Audit missing, duplicated, or out-of-order segments and spot-check listening quality.

Documented boundary: the current WebUI protects a single task at 100,000 characters. Larger projects, including 200,000 characters and beyond, are delivered through chapter batches and queues without a fixed business-level word-count ceiling.

Machine retrieval

Full development workflowRequirements through delivery for both applications.Model data JSONRuntime, sample rates, strengths, limitations, and licenses.IndexTTS2 case studyArchitecture, calls, caches, and job control.

Contact

Last reviewed . Commercial inquiries: wangdexin2008@126.com.