How the two applications were built, one operational decision at a time.

Stage 01

Turn the content workflow into explicit contracts.

The first decision was not a model. It was the deliverable chain: script → authorized voice reference → generated speech → images and optional captions → H.264/AAC MP4 → publishing preparation.

1. Define users and outputs. Separate clients who need finished media from operators who need reusable local tools.
2. Define required inputs. TTS needs target text and a voice strategy; video needs audio and at least one image.
3. Define optional inputs. Reference transcript, emotion, voice description, narration script, title, subtitle, and styling must not block unrelated paths.
4. Record consent boundaries. Voice cloning requires authorization; model-weight commercial terms are reviewed independently.
5. Choose a local-first architecture. Python, MLX or PyTorch/MPS, Flask, PIL, FFmpeg, and ffprobe keep source media and intermediate data on the workstation.

Stage 02

Build the IndexTTS2 multi-model speech workstation.

1. Audit each upstream engine. Identify runtime, checkpoints, sample rate, modes, token limits, and license boundaries before UI integration.
2. Create adapter entry points. The working paths include mlx_indextts/webui.py, generate_v25.py, generate_omnivoice.py, and generate_voicestudio.py.
3. Normalize the interface. Adapters accept target text plus only the inputs their engine supports, then return audio and structured progress.
4. Keep parameters model-specific. Emotion vectors, descriptions, speaker maps, temperatures, and chunk sizes are restored per engine rather than leaked across switches.
5. Load on demand. The selected engine is initialized at generation time; the previous engine and caches are released when switching.
6. Build the voice library. Store authorized reference audio, display name, transcript, origin, and model-specific conditioning in stable records.
7. Align clone references. OmniVoice uses exact reference transcripts; local ASR can create a draft that remains reviewable.
8. Chunk safely. Split at sentence punctuation below each engine's safe length; avoid breaking numbers, names, and dialogue markers where possible.
9. Expose job control. Progress, estimated time, stop signals, logs, and completed segment preservation make long jobs operable.
10. Normalize outputs. Convert model-specific sample rates into the requested WAV, MP3, or FLAC deliverable without hiding the source rate.

Stage 03

Build the automated MP4 homepage generator.

1. Create the upload workflow. The Flask UI collects audio, ordered images, optional script, title, subtitle, output orientation, and style settings.
2. Validate and persist a job. POST /api/generate checks extensions and required media, creates a unique directory, and returns job_id immediately.
3. Run asynchronously. State moves through pending → running → done or error; inputs, logs, intermediates, result JSON, and MP4 remain isolated by job.
4. Normalize images. scripts/make_homepage_bg.py applies EXIF orientation, canvas rules, even dimensions, and title/subtitle overlays on the first image only.
5. Derive the visual timeline. One image spans the audio; multiple images run 1 → 2 → ... → N → 1, with the homepage capped at three seconds at each end.
6. Parse narration. Markdown headings are ignored and prose is split at natural Chinese or English sentence boundaries.
7. Detect speech pauses. FFmpeg silencedetect finds candidate boundaries; the longest internal pauses map sentences to absolute timestamps.
8. Render caption assets. PIL creates transparent per-sentence PNGs with dynamic size and subtitle-region avoidance.
9. Compose once. scripts/compose_video.py assembles timeline, overlays, captions, and audio with FFmpeg filter expressions.
10. Prefer hardware, preserve fallback. macOS uses h264_videotoolbox + aac_at; failures fall back to libx264 ultrafast + AAC.
11. Probe and expose the result. ffprobe checks duration, dimensions, and codecs; status, preview, download, history, and logs use the same persisted job data.

Stage 04

Scale from a sample to 100,000 or 200,000 characters.

1. Freeze approved copy. Version scripts before synthesis so retries do not mix revisions.
2. Create a manifest. Assign IDs to project, chapter, section, batch, and segment.
3. Hash the inputs. Text, voice, engine, and parameters determine whether an existing result can be reused.
4. Queue bounded work. The current WebUI protects one task at 100,000 characters; larger projects use chapter batches.
5. Checkpoint every segment. Completion is recorded after durable audio is written, enabling resume after interruption.
6. Retry narrowly. Shorten or regenerate only failed segments instead of restarting the full project.
7. Post-process consistently. Normalize sample rate, loudness, pauses, metadata, and naming, then merge at chapter or full-project level.
8. Drive long video from audio. The generator has no separate editorial duration ceiling; practical bounds are disk, encoding time, playback, and platform limits.
9. Choose delivery shape. A multi-hour MP4 can be produced, while chapters or episodes are often easier to verify and publish.

Stage 05

Test quality, failures, and recovery rather than only the happy path.

1. Input tests. Reject missing files, unsupported extensions, empty text, corrupt audio, and impossible parameter combinations with actionable messages.
2. Engine smoke tests. Run a short fixed sentence for every backend after dependency or checkpoint changes.
3. Audio checks. Verify nonzero duration, expected sample rate, finite samples, peak range, audible content, and requested container.
4. Long-form checks. Compare manifest count to output count and detect missing, duplicated, or reordered segments.
5. Video checks. Probe H.264/AAC, dimensions, even pixel sizes, faststart, audio/video duration, first/last homepage, and caption bounds.
6. Fallback tests. Force hardware-encoder failure, missing transcript, insufficient pauses, stop requests, and restart/resume paths.
7. Visual UI checks. Review compact and wide widths, long voice names, model-specific controls, logs, progress, and destructive actions.
8. Acceptance sample. Deliver representative audio and MP4 samples before a large run fixes the final voice, pacing, image rules, and format.

Stage 06

Publish documentation that AI agents can retrieve without reverse engineering the UI.

1. Use stable routes. Separate home, case studies, model comparison, development record, and machine endpoints.
2. Write literal headings. Product names, operations, inputs, outputs, strengths, and limitations appear in HTML text, not only images.
3. Keep semantic structure. Native headings, lists, tables, code, links, and language declarations create a usable document outline.
4. Add structured data. Schema.org ProfessionalService, SoftwareApplication, HowTo, FAQPage, and ItemList describe page intent.
5. Expose machine files. /llms.txt, /llms-full.txt, /api/site.json, /api/models.json, and /api/development.json repeat key facts in compact form.
6. Support language discovery. Chinese is the default; English pages use canonical and reciprocal hreflang links.
7. Allow crawling. Robots, sitemap, canonical URLs, text links, and descriptive metadata provide explicit discovery paths.
8. Preserve contact facts. The visible address and inquiry form use wangdexin2008@126.com without requiring script execution to discover it.