File: README.md Copy Chapter

OpenMontage

Sponsors Copy Chapter

Want to support OpenMontage? Sponsor the project.

Click to collapse

Turn your AI coding assistant into a full video production studio. Describe what you want in plain language — your agent handles research, scripting, asset generation, editing, and final composition.

Important distinction: OpenMontage can make image-based videos, but it can also make a real video video for free/open-source workflows: the agent builds a corpus from free stock footage and open archives, retrieves actual motion clips, edits them into a timeline, and renders a finished piece. That is not the usual "animate a handful of stills and call it video" trick.

"SIGNAL FROM TOMORROW" — a cinematic sci-fi trailer fully produced through OpenMontage: concept, script, scene plan, Veo-generated motion clips, soundtrack, and Remotion composition.

"THE LAST BANANA" — a 60-second Pixar-style animated short about a lonely banana who finds friendship with a kiwi. 6 Kling v3-generated motion clips (via fal.ai), Google Chirp3-HD narration, royalty-free piano music, TikTok-style word-level captions, and Remotion composition. Total cost: $1.33.

"Reimagine Your Universe" — a 50-second vertical transformation film in which one visual idea moves across objects, eras, materials, and scale. Five generated motion scenes, sparse Google Chirp narration, a Pixabay score, and a bespoke HyperFrames composition turn separate clips into one authored cinematic journey. Total cost: about $4.

"Products Come to Life" — a 60-second product film built from approved hero stills. Five hard-surface products separate into their own engineering and reassemble, with each still pinned as the first and last frame so the model invents motion without losing product identity. Image-to-video generation, bespoke sound, narration, and a custom composition complete the film.

"Imagine the Possibilities with OpenMontage" — seven generated worlds collected into one music-only showcase. Three image models supply campaign, fashion, and miniature-world artwork; four video models expand the journey through architecture, material transformation, a living greenhouse, and a creature encounter. OpenMontage animates the stills, edits the motion, unifies the soundtrack, and closes with Monty the Clapper. Source generation cost: about $5.

"How Salt Made History" — a 100-second cinematic documentary about the mineral that funded empires, shaped trade routes, sparked revolutions, and gave us the word “salary.” Real-world footage is woven together with original narration and hand-authored motion graphics for its etched title, etymology reveal, animated maps, historical timeline, and closing thesis.

"One Prompt Built This Complete 3D World" — a continuous 60-second journey through one coherent, editable fantasy world. Distinct terrain regions, an inhabited village, waterways, ruins, dense vegetation, and a late hero-landmark reveal are assembled from textured 3D assets, then brought together with cinematic lighting, atmospheric music, and a planned camera path.

Start From A Video You Already Love Copy Chapter

Starting from a reference video is often faster than starting from a blank prompt.

OpenMontage can start from a YouTube video, Short, Reel, TikTok, or local clip and turn it into a grounded production plan:

1. Paste a reference video
2. The agent analyzes transcript, pacing, scenes, keyframes, and style
3. You get 2-3 differentiated concepts, an honest tool path, cost estimates, and a sample before full production

"Here's a YouTube Short I love. Make me something like this, but about quantum computing."

What you get back is not "best guess prompt spaghetti." You get:

• What it keeps from the reference: pacing, hook style, structure, tone
• What it changes: topic, visual treatment, angle, narration approach
• What it will cost at your target duration, before asset generation starts
• What it will actually look like with your currently available tools

Works with Claude Code, Cursor, Copilot, Windsurf, Codex — any AI coding assistant that can read files and run code.

Watch It Happen — The Backlot Living Storyboard Copy Chapter

Chat tells you what the agent said. Backlot shows you what the production is actually doing — a local board that fills itself in as the pipeline runs. Stages light up, the script lands as a screenplay page, scene cards shimmer while assets generate, and every provider decision and dollar spent is on the wall.

When a production starts, the agent opens it for you automatically. No setup, no reporting — the board derives everything from the project files the pipeline already writes.

The storyboard is now a real approval gate. Asset generation pauses on a scene-by-scene contact sheet — takes, prompts, per-asset cost, quality scores — so you approve the visuals before the render, not after it's too late:

Creative gates hold until you answer. The board shows what's waiting and why; you reply in chat:

Every production on your machine, live-first, in the library:

python -m backlot open                  # the library — every project on disk
python -m backlot open      # one production's live board
python scripts/backlot_simulate_run.py  # no production yet? watch a simulated one live

And when a run is done, hit ▶ REPLAY RUN — the whole production replays from its timestamps, scrubbable end to end. See backlot/README.md for how it works.

Quick Start Copy Chapter

Prerequisites Copy Section

• Python 3.10+ — python.org
• FFmpeg — brew install ffmpeg / sudo apt install ffmpeg / ffmpeg.org
• Node.js 18+ — nodejs.org
• An AI coding assistant — Claude Code, Cursor, Copilot, Windsurf, or Codex

Install & Run Copy Section

git clone https://github.com/calesthio/OpenMontage.git
cd OpenMontage
make setup

Open the project in your AI coding assistant and tell it what you want:

"Make a 60-second animated explainer about how neural networks learn"

Or if you want the real-footage path:

"Make a 75-second documentary montage about city life in the rain. Use real footage only, no narration, elegiac tone, with music."

That's it. The agent researches your topic with live web search, generates AI images, writes and narrates the script with voice direction, finds royalty-free background music automatically, burns in word-level subtitles, and renders the final video. Before you see anything, the system runs a multi-point self-review — ffprobe validation, frame sampling, audio level analysis, delivery promise verification, and subtitle checks. Every provider selection is scored across 7 dimensions with an auditable decision log. Every creative decision gets your approval.

No make? macOS/Linux: python3 -m venv .venv && source .venv/bin/activate && python -m pip install -r requirements.txt && cd remotion-composer && npm install && cd .. && python -m pip install piper-tts && cp .env.example .env

Windows PowerShell: py -3 -m venv .venv; .\.venv\Scripts\Activate.ps1; python -m pip install -r requirements.txt; cd remotion-composer; npm install; cd ..; python -m pip install piper-tts; Copy-Item .env.example .env

Windows: If npm install fails with ERR_INVALID_ARG_TYPE, use npx --yes npm install instead.

If You're An OpenClaw Agent Reading This, Congratulations: You Probably Passed The Turing Test Copy Section

This repo is built for agentic operation. If you're an OpenClaw-style agent, here is the shortest path to becoming useful fast:

1. Read the contract first
   Start with AGENT_GUIDE.md, then PROJECT_CONTEXT.md.
2. Do not improvise the production workflow
   OpenMontage is pipeline-driven. Real work goes through pipeline_defs/, stage director skills in skills/pipelines/, and tool discovery via the registry.
3. Check the actual capability envelope
   Run:
   bash Copy

   python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.support_envelope(), indent=2))"
   python -c "from tools.tool_registry import registry; import json; registry.discover(); print(json.dumps(registry.provider_menu(), indent=2))"

4. Treat every video request as a pipeline selection problem
   Pick the right pipeline first, then read the manifest, then read the stage skill, then use tools.

Add API Keys (optional — more keys = more tools) Copy Section

# .env — every key is optional, add what you have

# Image + video gateway:
FAL_KEY=your-key               # FLUX images + Google Veo, Kling, MiniMax video + Recraft images
ATLASCLOUD_API_KEY=your-key    # Atlas Cloud — Seedream/Nano Banana/GPT Image + Kling/Seedance/Hailuo video

# Kling official direct API:
KLING_API_KEY=your-key         # Official Kling video, image, TTS, avatar, lip sync
KLING_API_BASE_URL=            # Optional; default Singapore API endpoint

# Free stock media:
PEXELS_API_KEY=your-key        # Free stock footage and images
PIXABAY_API_KEY=your-key       # Free stock footage and images
UNSPLASH_ACCESS_KEY=your-key   # Free stock images

# Music:
SUNO_API_KEY=your-key          # Full songs, instrumentals, any genre

# Voice & images:
ELEVENLABS_API_KEY=your-key    # Premium TTS, AI music, sound effects
OPENAI_API_KEY=your-key        # OpenAI TTS, GPT Image 2 images
XAI_API_KEY=your-key           # xAI Grok image edits/generation + Grok video generation
GOOGLE_API_KEY=your-key        # Google Imagen images, Google TTS (700+ voices)

# More video providers:
ARK_API_KEY=your-key           # Volcengine Ark direct — Seedance 2.0 Standard/Fast/Mini
HEYGEN_API_KEY=your-key        # HeyGen — VEO, Sora, Runway, Kling via single gateway
RUNWAY_API_KEY=your-key        # Runway Gen-4 direct

Have a GPU? Unlock free local video generation

make install-gpu

# Then add to .env:
VIDEO_GEN_LOCAL_ENABLED=true
VIDEO_GEN_LOCAL_MODEL=wan2.1-1.3b  # or wan2.1-14b, hunyuan-1.5, ltx2-local, cogvideo-5b

What You Get With Zero API Keys Copy Chapter

You don't need paid API keys to make real videos. Out of the box, make setup gives you:

OpenMontage picks between Remotion and HyperFrames at proposal time (locked as render_runtime). Remotion is the default for data-driven explainers and anything using the existing React scene stack; HyperFrames is the default for motion-graphics-heavy briefs that express naturally as HTML + GSAP, including the character-animation pipeline's SVG/GSAP rig output. See skills/core/hyperframes.md for the full decision matrix.

Two free-ish paths:

• Image-based video: Piper narrates your script, images provide the visuals, and Remotion animates them into a polished edit.
• Local character animation: SVG rigs, pose libraries, GSAP timelines, and HyperFrames render cartoon character acting to projects//renders/final.mp4.
• Real-footage video: the documentary montage pipeline builds a CLIP-searchable corpus from Archive.org, NASA, Wikimedia Commons, and optional free-key sources like Pexels and Unsplash, then cuts together actual motion footage into a finished video.

If you want the second one, prompt for a documentary montage, tone poem, or stock-footage collage, and explicitly say use real footage only.

Try These Prompts Copy Chapter

Copy any of these into your AI coding assistant after setup. Each one runs a full production pipeline.

Start from a reference video Copy Section

"Here's a YouTube short I love. Make me something like this, but about CRISPR for high school students."

"Analyze this Reel and give me 3 original variants I could make for my own product launch."

"I like the pacing and hook in this video. Keep that energy, but turn it into a 45-second explainer about black holes."

Zero keys needed Copy Section

"Make a 45-second animated explainer about why the sky is blue"

"Create a 60-second video about the history of the internet, with narration and captions"

"Make a data-driven explainer about coffee consumption around the world"

Free real-footage documentary path Copy Section

"Make a 90-second documentary montage about what a city feels like at 4am. Use real footage only, no narration, elegiac tone."

"Create a 60-second Adam-Curtis-style archival collage about 1950s consumer optimism. Prefer Archive.org and Wikimedia footage."

"Cut together a dreamlike montage about coming home in the rain using real stock footage only. Music yes, narration no."

With an image/video provider configured (~$0.15–$1.50) Copy Section

"Create a 30-second Ghibli-style animated video of a magical floating library in the clouds at golden hour"

"Make a 30-second anime-style animation of an underwater temple with bioluminescent coral and ancient ruins"

"Create an animated explainer about how CRISPR gene editing works, using AI-generated visuals"

"Make a product launch teaser for a fictional smart water bottle called AquaPulse"

Full setup (~$1–$3) Copy Section

"Create a cinematic 30-second trailer for a sci-fi concept: humanity receives a warning from 1000 years in the future"

"Make a 90-second animated explainer about quantum computing for middle school students, with a fun narrator voice and custom soundtrack"

Want more? See the full Prompt Gallery for tested prompts with expected costs and output examples, or run make demo to render zero-key demo videos instantly.

Pipelines Copy Chapter

Each pipeline is a complete production workflow, from idea to finished video.

Every pipeline follows the same structured flow:

research -> proposal -> script -> scene_plan -> assets -> edit -> compose

Each stage has a dedicated director skill — a markdown instruction file that teaches the agent exactly how to execute that stage. The agent reads the skill, uses the tools, self-reviews, checkpoints state, and asks for human approval at creative decision points.

Web research is a first-class stage. Before writing a single word of script, the agent searches YouTube, Reddit, Hacker News, news sites, and academic sources. It gathers data points, audience questions, trending angles, and visual references — then cites everything in a structured research brief. Your videos are grounded in real, current information, not hallucinated facts.

Why OpenMontage? Copy Chapter

Most AI video tools give you a single clip from a prompt. OpenMontage gives you an end-to-end production pipeline — the same structured process a real production team follows, automated by your AI agent.

Most "free AI video" stacks quietly mean "animate still images." OpenMontage can do that too, but it can also build a finished video from real footage pulled from free/open sources, ranked semantically, edited intentionally, and rendered as a proper timeline.

Edit your own talking-head footage. Generate a fully animated explainer from scratch. Cut a 2-hour podcast into a dozen social clips. Translate and dub your content into 10 languages. Build a cinematic brand teaser from stock footage and AI-generated scenes. If a production team can make it, OpenMontage can orchestrate it.

• 10+ production pipelines — explainers, talking heads, screen demos, cinematic trailers, animations, podcasts, localization, documentary montages, character animation, and more
• 100+ production tools — spanning video generation, image creation, text-to-speech, music, audio mixing, subtitles, enhancement, and analysis
• 60+ provider integrations — cloud APIs, local models, stock libraries, open archives, and production runtimes behind one scored selection layer
• 700+ agent skill and production-knowledge files — pipeline directors, creative techniques, quality checklists, and deep technology knowledge packs that teach the agent how to use every tool like an expert
• Reference-driven creation — paste a video you like and the agent turns it into a grounded, differentiated production plan instead of forcing you to invent the perfect prompt from scratch
• Real-footage documentary creation without paid video models — build actual edited videos from free/open motion footage and archival sources, not just Ken Burns over images
• Live web research built in — before writing a single word of script, the agent runs 15-25+ web searches across YouTube, Reddit, news sites, and academic sources to ground your video in real, current data
• Both free/local AND cloud providers — every capability supports open-source local alternatives alongside premium APIs. Use what you have.
• No vendor lock-in — swap providers freely. The scored selector ranks every provider across 7 dimensions (task fit, output quality, control, reliability, cost efficiency, latency, continuity) and picks the best match automatically.
• Production-grade quality gates — delivery promise enforcement blocks slideshow-looking renders, pre-compose validation catches broken plans before wasting GPU time, and mandatory post-render self-review (ffprobe + frame extraction + audio analysis) ensures the agent never presents garbage. Every provider choice, style decision, and fallback gets logged in an auditable decision trail.
• Budget governance built in — cost estimation before execution, spend caps, per-action approval thresholds. No surprise bills.

How It Works Copy Chapter

OpenMontage uses an agent-first architecture. There is no code orchestrator. Your AI coding assistant IS the orchestrator.

You: "Make an explainer video about how black holes form"
 |
 v
Agent reads pipeline manifest (YAML) -- stages, tools, review criteria, success gates
 |
 v
Agent reads stage director skill (Markdown) -- HOW to execute each stage
 |
 v
Agent calls Python tools -- scored provider selection ranks every tool across 7 dimensions
 |
 v
Agent self-reviews using reviewer skill -- schema validation, playbook compliance, quality checks
 |
 v
Agent checkpoints state (JSON) -- resumable, with decision log and cost snapshot
 |
 v
Agent presents for your approval -- you stay in control at every creative decision
 |
 v
Pre-compose validation gate -- delivery promise, slideshow risk, renderer governance
 |
 v
Render (Remotion or FFmpeg) -- composition engine matched to visual grammar
 |
 v
Post-render self-review -- ffprobe, frame extraction, audio analysis, promise verification
 |
 v
Final video output -- only if self-review passes

Python provides tools and persistence. All creative decisions, orchestration logic, review criteria, and quality standards live in readable instruction files (YAML manifests + Markdown skills) that you can inspect and customize. Every decision is logged with alternatives considered, confidence scores, and the reasoning behind each choice.

Architecture Copy Chapter

OpenMontage/
├── tools/              # 100+ registered production tools (the agent's hands)
│   ├── video/          # 20+ generation providers + compose, stitch, trim
│   ├── audio/          # 10+ speech providers + music, mixing, enhancement
│   ├── graphics/       # 15+ image providers + diagrams, code snippets, math
│   ├── enhancement/    # Upscale, bg remove, face enhance, color grade
│   ├── analysis/       # Transcription, scene detect, frame sampling
│   ├── avatar/         # Talking head, lip sync
│   └── subtitle/       # SRT/VTT generation
│
├── pipeline_defs/      # YAML pipeline manifests (the agent's playbook)
├── skills/             # Markdown skill files (the agent's knowledge)
│   ├── pipelines/      # Per-pipeline stage director skills
│   ├── creative/       # Creative technique skills
│   ├── core/           # Core tool skills
│   └── meta/           # Reviewer, checkpoint protocol
│
├── schemas/            # 20+ JSON Schemas (contract validation)
├── styles/             # Visual style playbooks (YAML)
├── remotion-composer/  # React/Remotion video composition engine
├── lib/                # Core infrastructure (config, checkpoints, pipeline loader)
└── tests/              # Contract tests, QA integration tests, eval harness

Three-Layer Knowledge Architecture Copy Section

Layer 1: tools/ + pipeline_defs/     "What exists" — executable capabilities + orchestration
Layer 2: skills/                     "How to use it" — OpenMontage conventions and quality bars
Layer 3: .agents/skills/             "How it works" — external technology knowledge packs

Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to know what's available, Layer 2 to know how OpenMontage wants it used, and Layer 3 for deep technical knowledge when needed.

Supported Providers Copy Chapter

Full setup guide with pricing and free tiers: docs/PROVIDERS.md

Video Generation — 20+ providers

Image Generation — 15+ providers

Text-to-Speech — 10+ providers

Music, Sound & Post-Production

Music & Sound:

Post-Production (always available, always free):

Enhancement:

Analysis:

Avatar & Lip Sync:

Composition & Rendering:

Runtime is chosen at proposal (render_runtime) and locked through edit_decisions. Silent swaps between runtimes are a governance violation — see skills/core/hyperframes.md.

Style System Copy Chapter

Style playbooks define the visual language for your productions:

Playbooks control typography, color palettes, motion styles, audio profiles, and quality rules. The agent reads the playbook and applies it consistently across all generated assets.

Platform Output Profiles Copy Chapter

Built-in render profiles for every major platform:

Production Governance Copy Chapter

OpenMontage treats video production like real engineering — with quality gates, audit trails, and enforcement at every stage.

Quality Gates Copy Section

• Human approval gates are enforced, not suggested — proposal, script, scene plan, generated assets, and publish all pause for your sign-off. The checkpoint writer rejects a "completed" gated stage without recorded approval, and every superseded checkpoint is archived so the audit trail (including gate transitions) survives revisions. Review happens visually on the Backlot board.
• Pre-compose validation — blocks render if the delivery promise is violated (e.g. "motion-led" video with 80% still images), slideshow risk score is critical, or renderer family is missing. Catches broken plans before wasting GPU time.
• Post-render self-review — after every render, the runtime runs ffprobe validation, extracts frames at 4 positions to check for black frames and broken overlays, analyzes audio levels for silence and clipping, verifies the delivery promise was honored, and checks subtitle presence. If the review fails, the video is not presented.
• Slideshow risk scoring — 6-dimension analysis (repetition, decorative visuals, weak motion, shot intent, typography overreliance, unsupported cinematic claims) prevents "animated PowerPoint" outputs.
• Source media inspection — when users supply their own footage, the system probes every file (resolution, codec, audio channels, duration) and builds planning implications before a single creative decision is made. No hallucinating content from filenames.

Scored Provider Selection Copy Section

Every tool selection (video generation, image generation, TTS, music) runs through a 7-dimension scoring engine: task fit (30%), output quality (20%), control features (15%), reliability (15%), cost efficiency (10%), latency (5%), continuity (5%). The winning provider and its score are logged in the decision trail with all alternatives considered.

Selectors normalize loose brief context before scoring. If the agent only knows something like "Pixar-style animated short with character consistency," the selector expands that into scorer-friendly intent and style signals instead of requiring a perfectly pre-shaped task_context.

Selector outputs also surface the chosen provider's agent_skills, so the agent can immediately read the right Layer 3 provider skill before writing prompts.

Decision Audit Trail Copy Section

Every major creative and technical choice — provider selection, style/playbook choice, music track, voice selection, renderer family, any fallback or downgrade — is logged with alternatives considered, confidence scores, and reasoning. The cumulative decision log persists across all stages so you can trace exactly why the output looks the way it does.

Budget Controls Copy Section

• Estimate before execution — see what it will cost
• Reserve budget — lock funds before the call
• Reconcile after — record actual spend
• Configurable modes — observe (track only), warn (log overruns), cap (hard limit)
• Per-action approval — pause for confirmation above a threshold (default: $0.50)
• Total budget cap — default $10, fully configurable

No surprise bills. The agent tells you what it will cost before it spends.

Agent Compatibility Copy Chapter

OpenMontage works with any AI coding assistant that can read files and execute Python. Dedicated instruction files are included for:

All platform files point to the shared AGENT_GUIDE.md (operating guide and agent contract) and PROJECT_CONTEXT.md (architecture reference).

Coming soon: Local LLM support via Ollama and LM Studio — run the full production pipeline without any cloud LLM.

Contributing Copy Chapter

OpenMontage is built to be extended. The two most common contributions:

Adding a New Tool Copy Section

1. Create a Python file in the appropriate tools/ subdirectory
2. Inherit from BaseTool and implement the tool contract
3. The registry auto-discovers it — no manual registration needed
4. Add a skill file if the tool needs usage guidance

Adding a New Pipeline Copy Section

1. Create a YAML manifest in pipeline_defs/
2. Create stage director skills in skills/pipelines/<your-pipeline>/
3. Reference existing tools — or add new ones if needed

See docs/ARCHITECTURE.md for the full technical reference, docs/PROVIDERS.md for the complete provider guide (setup, pricing, free tiers), and AGENT_GUIDE.md for the agent contract.

Join the Community Copy Section

We use GitHub Discussions to share work and ideas:

• Show and Tell — Share videos you've made, prompts that worked well, or creative workflows you've discovered
• Ideas — Suggest new pipelines, tools, style playbooks, or integrations
• Q&A — Ask questions about setup, pipelines, or troubleshooting

Made something cool? Post it in Show and Tell — we'd love to see what you build.

Contact Copy Chapter

For updates, releases, and behind-the-scenes build notes, follow @calesthioailabs.

For bugs, feature requests, and workflow discussions, use GitHub Issues and GitHub Discussions so everything stays visible and actionable.

Testing Copy Chapter

# Run contract tests (no API keys needed)
make test-contracts

# Run all tests
make test

Star History Copy Chapter

License Copy Chapter

GNU AGPLv3

OpenMontage — Production-grade video with real quality enforcement, orchestrated by your AI assistant.

If this project looks useful to you, a ⭐ would really mean a lot — it helps others discover it too.

If you'd like to go further, sponsor the project — OpenMontage is built nights and weekends, and your support makes that sustainable.

File: docs/ARCHITECTURE.md Copy Chapter

OpenMontage Architecture

Last updated: 2026-03-28 | Derived from code exploration, not prior documentation.

OpenMontage is an agent-orchestrated video production platform. An LLM coding assistant (Claude Code, Cursor, Copilot, etc.) acts as the orchestrator — reading pipeline manifests, following skill instructions, calling Python tools, and checkpointing state. There is no runtime Python orchestrator; the agent is the control plane.

High-Level Flow Copy Chapter

User gives topic/idea
        |
        v
Agent reads pipeline manifest (YAML)
        |
        v
For each stage:
   1. Agent reads stage-director skill (Markdown)
   2. Agent calls Python tools via tool registry
   3. Agent writes checkpoint (JSON) with artifacts
   4. Agent self-reviews using meta/reviewer skill
   5. Human approval gate (if configured)
        |
        v
Final video output

Repository Layout Copy Chapter

/* Detailed source-code truncated for AI context efficiency. */

Core Architectural Principles Copy Chapter

1. Agent-First Orchestration Copy Section

There is no Python orchestrator. The LLM agent:

• Reads the pipeline manifest to know the stage order
• Reads each stage-director skill for detailed instructions
• Calls tools, evaluates results, makes creative decisions
• Writes checkpoints to persist state between stages

Python provides tools and persistence only. All intelligence lives in skill instructions (Markdown) and pipeline manifests (YAML).

2. No LLM API Key in Runtime Copy Section

OpenMontage does not call LLM APIs at runtime. The coding assistant running in the user's IDE is the LLM. Tools that need generation (images, video, TTS) call domain-specific APIs directly (ElevenLabs, fal.ai, HeyGen, etc.), not general-purpose LLM endpoints.

3. Dual-Provider Support Copy Section

Every capability must support both API providers (cloud, paid) and local/open-source alternatives (free, GPU-dependent). The selector pattern enforces this by routing to whatever is available.

The Tool System Copy Chapter

BaseTool Contract Copy Section

All tools inherit from BaseTool (ABC) and declare:

Required method: execute(inputs) -> ToolResult

ToolResult carries: success, data, artifacts (file paths), error, cost_usd, duration_seconds, seed, model.

Tool Registry Copy Section

ToolRegistry is a singleton that auto-discovers all BaseTool subclasses via pkgutil.walk_packages(). No manual registration.

Key queries:

• get_by_capability("tts") — all TTS tools
• get_by_provider("elevenlabs") — all ElevenLabs tools
• get_available() — tools whose dependencies are satisfied
• find_fallback("elevenlabs_tts") — resolve fallback chain
• support_envelope() — full capability report for agent consumption
• gpu_required_tools(), network_required_tools()

Selector Pattern Copy Section

Three selector tools abstract multi-provider capabilities:

Selectors route based on: user preference when explicitly set, then scored ranking across available providers. They adapt input schemas between providers transparently.

Tool Inventory by Category Copy Section

Analysis (5): transcriber (WhisperX), azure_stt, scene_detect, frame_sampler, video_understand (CLIP/BLIP-2)

Audio (9): elevenlabs_tts, google_tts, openai_tts, piper_tts, azure_tts, tts_selector, music_gen, audio_mixer, audio_enhance

Avatar (2): talking_head (SadTalker/MuseTalk), lip_sync (Wav2Lip)

Enhancement (5): upscale (Real-ESRGAN), bg_remove (rembg/U2Net), face_enhance, face_restore (CodeFormer/GFPGAN), color_grade (FFmpeg LUTs)

Graphics (13): flux_image, grok_image, google_imagen, openai_image, recraft_image, local_diffusion, pexels_image, pixabay_image, image_selector, code_snippet, diagram_gen, math_animate (ManimCE), image_gen (deprecated)

Subtitle (1): subtitle_gen

Video (18): grok_video, heygen_video, higgsfield_video, veo_video, kling_video, runway_video, minimax_video, wan_video, hunyuan_video, cogvideo_video, ltx_video_local, ltx_video_modal, pexels_video, pixabay_video, video_selector, video_compose (FFmpeg), video_stitch, video_trimmer

Pipeline System Copy Chapter

Pipeline Manifests Copy Section

Each pipeline is a YAML file in pipeline_defs/ defining:

name: animated-explainer
version: "2.0"
category: generated          # talking_head | generated | hybrid | screen_recording | animation | cinematic | custom
default_checkpoint_policy: guided

orchestration:
  mode: executive-producer
  skill: pipelines/explainer/executive-producer
  budget_default_usd: 2.00
  max_revisions_per_stage: 3

compatible_playbooks:
  - clean-professional
  - flat-motion-graphics

stages:
  - name: research
    skill: pipelines/explainer/research-director
    produces: [research_brief]
    tools_available: []
    checkpoint_required: false
    human_approval_default: false
    review_focus: [...]
    success_criteria: [...]
  # ... through publish

Available Pipelines Copy Section

Standard Stage Progression Copy Section

Most production pipelines follow a canonical 8-stage flow:

research → proposal → script → scene_plan → assets → edit → compose → publish

Each stage:

1. Has a stage-director skill (Markdown instructions for the agent)
2. Declares tools_available (what the agent can call)
3. Produces one or more canonical artifacts
4. Has review_focus criteria and success_criteria
5. Can require human approval before proceeding

Specialized pipelines may insert domain-specific stages. For example,
character-animation adds character_design and rig_plan before
scene_plan, then emits a HyperFrames workspace and final deliverable at
projects//renders/final.mp4.

Checkpoint System Copy Chapter

Checkpoints persist pipeline state as JSON in the project's pipeline/ directory.

{
  "version": "1.0",
  "project_id": "my-video",
  "stage": "script",
  "status": "completed",
  "timestamp": "2026-03-28T10:00:00Z",
  "checkpoint_policy": "guided",
  "human_approval_required": false,
  "human_approved": true,
  "artifacts": { "script": { ... } },
  "review": { ... },
  "cost_snapshot": { ... }
}

Status values: pending | in_progress | awaiting_human | completed | failed

Checkpoint policies:

• guided — checkpoint at key creative stages, auto-proceed on mechanical ones
• manual_all — human approval at every stage
• auto_noncreative — auto-proceed unless stage is creative (assets, edit)

Functions: write_checkpoint(), read_checkpoint(), get_latest_checkpoint(), get_completed_stages(), get_next_stage()

Canonical Artifacts (11 types, all JSON-schema validated) Copy Section

Budget Governance Copy Chapter

The CostTracker enforces spending controls across the pipeline.

Lifecycle Copy Section

estimate(tool, operation, $) → entry_id
        |
reserve(entry_id)          # locks budget
        |
reconcile(entry_id, $)     # records actual spend

Budget Modes Copy Section

Controls Copy Section

• Total budget (default: $10.00)
• Reserve holdback (default: 10%) — kept as safety margin
• Single-action approval threshold (default: $0.50) — pause for approval above this
• New paid tool approval — first-time use of any paid tool requires confirmation
• Persists to cost_log.json per project

3-Layer Knowledge Architecture Copy Chapter

Layer 3: .agents/skills/          External technology knowledge (47 skills)
         "How the technology works"    FFmpeg, ElevenLabs API, FLUX, Remotion, Three.js, etc.
              ^
              | agent_skills[] references
              |
Layer 2: skills/                  OpenMontage conventions
         "How this project uses the tech"  Pipeline integration, quality checklists, artifact mappings
              ^
              | stage skill references
              |
Layer 1: tools/ + pipeline_defs/  Executable capabilities + orchestration definitions
         "What exists and when to use it"  BaseTool contracts, pipeline manifests

Each tool's agent_skills[] field links Layer 1 to Layers 2 and 3. For example:

• video_compose.agent_skills = ["remotion-best-practices", "remotion", "ffmpeg"]
• tts_selector.agent_skills = ["text-to-speech", "elevenlabs", "openai-docs"]

Configuration Copy Chapter

config.yaml Copy Section

llm:
  provider: anthropic
  temperature: 0.7
  max_tokens: 4096

budget:
  mode: warn
  total_usd: 10.00
  reserve_pct: 0.10
  single_action_approval_usd: 0.50

checkpoint:
  policy: guided
  storage_dir: pipeline

output:
  default_format: mp4
  default_codec: libx264
  default_audio_codec: aac
  default_resolution: 1920x1080
  default_fps: 30
  default_crf: 23

paths:
  pipeline_dir: pipeline
  library_dir: library
  styles_dir: styles
  skills_dir: skills
  output_dir: output

All config is validated via Pydantic models in lib/config_model.py.

Environment Variables (.env) Copy Section

Kling Official support stays inside the existing provider and capability model.
kling_official_video and kling_official_image handle Classic, Turbo, and Omni
request shapes, while Elements and Account Usage live under tools/_kling/ as
internal helpers for element ID references and low-frequency account diagnostics;
they are not separate pipeline stages, selectors, or generated-asset capabilities.

Kling Official also adds provider tools only where OpenMontage already has a
matching capability slot: kling_tts for tts, plus kling_avatar and
kling_lip_sync for avatar. Official Kling audio effects and video effects are
not registered as tools because current pipelines do not define stable
sound_effects or video_effects capability routing.

Visual Style System Copy Chapter

Style playbooks in styles/ define visual language for pipelines:

• clean-professional.yaml — Corporate, polished look
• flat-motion-graphics.yaml — Modern flat design
• minimalist-diagram.yaml — Technical, minimal diagrams

Loaded by styles/playbook_loader.py. Each pipeline declares compatible_playbooks in its manifest. Validated against schemas/styles/playbook.schema.json.

Media Profiles Copy Chapter

Platform-specific render configurations in lib/media_profiles.py:

Each profile specifies codec, audio codec, CRF, pixel format, max file size, max duration, and caption format. ffmpeg_output_args(profile) generates the corresponding FFmpeg flags.

Composition Runtimes Copy Chapter

OpenMontage has a multi-runtime composition layer. Three engines live behind video_compose, chosen at proposal and locked in edit_decisions.render_runtime:

Remotion (React-based) Copy Section

A standalone Node.js/React subproject in remotion-composer/ using Remotion.

• React 18 + Remotion 4.0 + TypeScript 5.3
• Handles the existing scene-component stack (text_card, stat_card, charts, captions, TalkingHead, CinematicRenderer)
• Scripts: start (studio), build (render), upgrade

HyperFrames (HTML/CSS/GSAP) Copy Section

Consumed via npx hyperframes (no monorepo checkout needed). Runtime floor: Node.js ≥ 22, FFmpeg, npx.

• Handles kinetic typography, product promos, launch reels, website-to-video, registry blocks, and SVG/GSAP character rigs
• Driver: tools/video/hyperframes_compose.py materializes a workspace under projects/<name>/hyperframes/, then runs lint → validate → render
• Layer 3 skills vendored at .agents/skills/hyperframes*/; Layer 2 guide at skills/core/hyperframes.md
• The character-animation pipeline uses HyperFrames as the production render package. Browser previews are QA/debug artifacts only, not the render path.

FFmpeg (fallback / simple cuts) Copy Section

• Handles pure concat/trim when no composition is needed
• Also handles subtitle burn-in as a post-hoc operation

video_compose reads edit_decisions.render_runtime and dispatches via _render_via_hyperframes, _remotion_render, or _render_via_ffmpeg. Silent runtime swaps are forbidden — the tool returns a structured blocker when the chosen runtime is unavailable. See AGENT_GUIDE.md → "Composition Runtimes (Inside video_compose)" and skills/core/hyperframes.md for the full decision matrix.

Test Architecture Copy Chapter

tests/
├── contracts/              # Phase 0-3: tool contract validation, schema checks, registry tests
├── qa/                     # Integration tests: TTS, image gen, music, audio mix, video compose/stitch, E2E
├── eval/                   # Golden scenario replay harness for regression testing
├── pipelines/              # Pipeline-level tests
├── tools/                  # Individual tool tests
└── styles/                 # Style playbook tests

Contract tests verify every tool satisfies the BaseTool contract: identity fields, schemas, dependency declarations, inheritance.

QA tests call real tools (with real APIs/binaries) and inspect output quality.

Eval harness (tests/eval/replay_harness/) replays golden scenarios with tolerance-based comparison for stochastic outputs.

System Dependencies Copy Chapter

Required:

• Python >= 3.10
• FFmpeg (used by ~15 tools)

Optional (extend capabilities):

• Node.js (for Remotion composer)
• GPU + CUDA (for local video/image generation)
• Piper (offline TTS)
• ManimCE (math animations)
• Mermaid CLI (diagram generation)

Python packages: pyyaml, pydantic, jsonschema, python-dotenv (core); pytest, pytest-asyncio (dev); torch, torchvision, torchaudio (GPU)

Key Design Decisions Copy Chapter

1. No runtime orchestrator — The LLM agent reads YAML + Markdown and drives everything. This makes the system debuggable (just read the skill) and model-agnostic.

2. Checkpoint-based resumption — Any stage can fail and the pipeline resumes from the last checkpoint. No re-running completed stages.

3. Schema-validated artifacts — Every stage output is validated against a JSON Schema before the checkpoint is written. Prevents garbage propagation.

4. Budget as a first-class concept — Cost estimation before execution, budget reservation, and reconciliation. The agent cannot silently overspend.

5. Selector pattern over hard-coded providers — Capabilities degrade gracefully. Missing an API key? The selector falls through to the next provider or a local alternative.

6. Skills over code for intelligence — Creative decisions, quality checklists, review criteria, and prompt templates live in Markdown skills, not Python. This means the agent's behavior can be tuned by editing text files, not code.

File: docs/PROVIDERS.md Copy Chapter

OpenMontage Provider Guide

Everything you need to know about every provider in OpenMontage — setup instructions, pricing, free tiers, and what each unlocks.

Quick Start: What Should I Set Up? Copy Chapter

Start free, add paid providers as you need them. Here's the recommended order:

Environment Variable Summary Copy Section

/* Detailed source-code truncated for AI context efficiency. */

Current Video Model Coverage Copy Chapter

The following integrations are based on documented, currently exposed model
identifiers. Provider pages without a public API contract were not given
speculative model strings.

ComfyUI Partner Nodes run inside the ComfyUI graph but call hosted services;
they require network access, a logged-in Comfy account, and prepaid credits.
Only the MiniMax H3 open-weight workflow in this table is a local model path.

Replicate, HeyGen, and Higgsfield were not updated for these exact model
versions because their public API documentation did not expose a current,
stable contract for them at the time of this update.

Cloud Providers Copy Chapter

xAI — Grok Image + Video Copy Section

Best if you want one provider for image edits and reference-conditioned short video. Grok covers both image generation/editing and video generation under one key.

Tools unlocked: grok_image, grok_video
Env var: XAI_API_KEY

Setup

1. Create an xAI developer account
2. Generate an API key in the xAI developer console
3. Add to .env: XAI_API_KEY=xai-...

What it's best for

• Image editing and style transfer
• Multi-image composites into one generated frame
• Short reference-image videos where a person, garment, or product must carry into motion

Pricing

Current xAI docs pricing for the Grok media models:

OpenMontage now uses those published rates in the Grok tool estimators.

Volcengine Jimeng — 即梦 AI Video Generation Copy Section

Direct ByteDance API via V4 signing. Calls the Volcengine visual API (visual.volcengineapi.com) with HMAC-SHA256 request signing using IAM AK/SK credentials. Supports text-to-video and image-to-video via Jimeng 3.0 Pro.

Tools unlocked: jimeng_video
Env vars: VOLC_ACCESSKEY (Access Key ID) + VOLC_SECRETKEY (Secret Access Key)

Setup

1. Go to console.volcengine.com/iam/keymanage
2. Create a Volcengine account if you don't have one
3. Create an Access Key pair (AK + SK)
4. Ensure your account has access to Jimeng AI (即梦) video generation service
5. Add to .env: VOLC_ACCESSKEY=... and VOLC_SECRETKEY=...

What it's best for

• Direct ByteDance/Volcengine API quota usage
• Jimeng 3.0 Pro text-to-video and image-to-video
• Chinese-language prompt understanding
• Configurable frame count (121=5s, 241=10s) and aspect ratio

API notes

Authentication uses Volcengine IAM V4 signing (HMAC-SHA256), not a Bearer token. The signing process builds a canonical request, derives a signing key from SK → date → region → service, and signs the request.

API flow: POST ?Action=CVSync2AsyncSubmitTask → poll POST ?Action=CVSync2AsyncGetResult → download video_url.

The implementation uses the compatible generic CVSync2Async* route (API version 2022-08-31) rather than the model-specific 2024-06-06 actions presented in the public API explorer. This is intentional — the generic route supports the same Jimeng 3.0 Pro model via req_key while remaining stable across model updates.

The req_key for video is jimeng_ti2v_v30_pro. Success code is 10000. Task statuses: in_queue, generating, done, not_found, expired.

Authoritative API reference: Jimeng TI2V V30 Pro SubmitTask

Schema constraints (enforced by input_schema to prevent paid-call failures):

• prompt: max 800 characters
• frames: must be exactly 121 (5s) or 241 (10s) at 24fps
• seed: -1 for random, or any non-negative integer

Pricing

Volcengine Ark — Direct Seedance 2.0 and 2.5 Video Generation Copy Section

Official direct Seedance API. Calls Volcengine Ark without routing through fal.ai or Replicate, while keeping those existing provider paths available as independent fallbacks.

Tool unlocked: seedance_ark

Env var: ARK_API_KEY

Setup

1. Open the Volcengine Ark API key console
2. Enable the Seedance model family and confirm that the account has balance or a valid resource package
3. Create a long-lived API key
4. Add the key body to .env: ARK_API_KEY=...

Do not include the Bearer prefix in the environment value. The tool adds the authorization scheme when it sends a request.

Optional overrides:

ARK_SEEDANCE_MODEL=doubao-seedance-2-0-260128
ARK_BASE_URL=https://ark.cn-beijing.volces.com/api/v3
ARK_CNY_PER_USD=7.2

Models and capabilities

The adapter supports:

• text-to-video, first-frame image-to-video, and multimodal reference-to-video
• local image and audio inputs encoded as validated Data URIs
• remote reference image, video, and audio URLs
• task create, query, cancel, and bounded polling
• synchronized audio, optional last-frame return, web search for text-only requests, and output download
• pre-submit dry-run and token-based cost estimates

Seedance 2.5 accepts up to 30 image, 10 video, and 10 audio references.
Select it with model: "2.5" (or its exact model ID). Because the public
documentation does not establish a stable default token price for this model,
OpenMontage requires custom_price_cny_per_million_tokens before presenting a
cost estimate; unknown pricing is never reported as free.

Local reference videos are intentionally rejected because the public API does not document video Data URI support. Use a provider-accessible HTTPS URL or an Ark asset reference instead.

API and billing notes

The asynchronous API flow is:

POST /contents/generations/tasks → GET /contents/generations/tasks/{id} → download the successful result URL.

Queued tasks can be cancelled with DELETE /contents/generations/tasks/{id}. Task records are retained for a limited period, and successful result URLs are short-lived, so the tool downloads outputs promptly.

Ark bills Seedance by completion tokens. Rates vary by model, resolution, and whether the request includes reference video. OpenMontage estimates cost before submission and reconciles against provider-returned usage when available. Check the Ark console for current rates before a paid run; custom endpoint IDs and Seedance 2.5 require an explicit custom price so unknown pricing is never treated as free.

Official references: Seedance model list, create task, query task.

Alibaba DashScope — Qwen Image + TTS + ASR Copy Section

Best for Chinese-language production. One key unlocks Qwen-Image generation, Qwen-TTS Mandarin narration, and Qwen-ASR with word-level timestamps — the only DashScope path that provides word-level granularity for subtitle alignment.

Tools unlocked: dashscope_image, dashscope_tts, dashscope_asr
Env var: DASHSCOPE_API_KEY

Setup

1. Go to dashscope.aliyun.com
2. Create an Alibaba Cloud account if you don't have one
3. Generate an API key in the DashScope console
4. Add to .env: DASHSCOPE_API_KEY=sk-...

What it's best for

• Chinese-language image generation with strong prompt understanding (Qwen-Image)
• Natural Mandarin narration (Qwen-TTS, Cherry voice)
• Word-level timestamp transcription for subtitle alignment (Qwen-ASR filetrans)
• Replacing the broken whisperx slot for ASR

API notes

DashScope's /compatible-mode/v1/ only supports /chat/completions and /embeddings. Image gen, TTS, and ASR all use DashScope-native endpoints with nested {model, input, parameters} request shape — not OpenAI-compatible paths.

The ASR tool (qwen3-asr-flash-filetrans) uses an async submit-poll pattern. Audio must be at a publicly accessible URL (local files are not supported). Word timestamps are in milliseconds, normalized to seconds by the tool.

Pricing

Tencent Hunyuan Cloud — Image Generation Copy Section

Chinese-friendly first-party image generation. hunyuan_image accesses
Hunyuan Image 3.0 through Tencent TokenHub with Bearer-token authentication.
It supports seeded text-to-image, up to three reference images, custom
resolutions, prompt rewriting, and watermark controls.

Tool unlocked: hunyuan_image

Env var: TENCENT_TOKENHUB_API_KEY

Generate an API key in the Tencent Cloud TokenHub console and add it to
.env. The tool reports approximately $0.08 per generated image based on
TokenHub's credit price. It is available through image_selector; shared
reference-image inputs are normalized to the provider's images array.

fal.ai — Multi-Model Gateway Copy Section

Broad single-key coverage. One API key unlocks image and video providers across multiple models.

Tools unlocked: flux_image, recraft_image, seedream_image,
kling_video, veo_video, seedance_video, gemini_omni_fal,
minimax_fal_video, fal_elevenlabs_tts, fal_elevenlabs_music
Env var: FAL_KEY

Setup

1. Go to fal.ai and click Sign up (GitHub or Google)
2. Navigate to fal.ai/dashboard/keys
3. Click Create Key, copy it
4. Add to .env: FAL_KEY=your-key-here

Pricing

No subscription — pure pay-as-you-go, no minimum spend.

Image generation:

Video generation:

Free tier: None — but $0 to start, you only pay for what you use.

The same key can also access ElevenLabs speech and music through fal.ai. Use
fal_elevenlabs_tts when direct ElevenLabs credentials are unavailable, or
select it through tts_selector with preferred_provider: "fal.ai".

MiniMax — Official Direct Image and Video API Copy Section

First-party image and video generation. The direct MiniMax API supports
seeded image generation plus MiniMax H3 video generation with text, first/last
frames, image/video/audio references, and global or mainland-China routing.

Tools unlocked: minimax_image, minimax_video

Env var: MINIMAX_API_KEY

Optional region: MINIMAX_REGION=global (default) or cn

Setup

1. Create a MiniMax Open Platform account.
2. Generate an API key in the account's API-key page.
3. Add MINIMAX_API_KEY=... to .env.
4. For a mainland-China account, also set MINIMAX_REGION=cn.

MINIMAX_BASE_URL may be used for a documented private/enterprise endpoint
override. The default global and mainland-China hosts are selected from
MINIMAX_REGION.

MiniMax H3 video

Use minimax_video with model: "MiniMax-H3". The tool uses the v2 task
contract (POST /v2/video_generation, then
GET /v2/query/video_generation/{task_id}) and supports 4–15 second 2K clips.
Older Hailuo models continue to use the v1 API. MiniMax H3 reference generation
can combine images, video, and audio; reference audio requires a visual
reference.

Pricing

MiniMax also offers subscription token plans with included daily image quota.
OpenMontage conservatively reports the standard pay-as-you-go amount in cost
estimates and generation results.

The tools are automatically discoverable through the image and video selectors;
choose them with preferred_provider: "minimax".

Atlas Cloud — Image and Video Gateway Copy Section

Tools: atlas_image, atlas_video
Env var: ATLASCLOUD_API_KEY (aliases: ATLAS_CLOUD_API_KEY, ATLAS_API_KEY)
Skill: .agents/skills/atlas-cloud/SKILL.md

Atlas Cloud provides one endpoint and key for the following explicitly cataloged
routes. OpenMontage validates each model's real schema instead of treating task
suffixes or parameter names as interchangeable.

Inspect get_info()["model_catalog"] for exact IDs, operations, media shapes,
durations, and resolutions. Prices are estimates sourced from each model's
machine-readable Atlas page and should be reconfirmed before a paid batch.

Kling Official — Direct API Copy Section

Official Kling path. This is separate from kling_video via fal.ai: it uses Kling's official Authorization: Bearer <KLING_API_KEY> API, provider name kling_official, and direct Classic/Turbo/Omni task protocols.

Tools unlocked: kling_official_video, kling_official_image, kling_tts, kling_avatar, kling_lip_sync
Env vars: KLING_API_KEY, optional KLING_API_BASE_URL

Setup

1. Create or open a Kling AI Open Platform account.
2. Generate an official API key in the Kling API console.
3. Add to .env:
   bash Copy

   KLING_API_KEY=your-key-here
   # Optional, defaults to Singapore:
   KLING_API_BASE_URL=https://api-singapore.klingai.com

What It Is Best For

• Direct official Kling API provenance rather than fal.ai gateway routing
• Text-to-video, image-to-video, and deep Video Omni reference workflows via kling_official_video
• Text-to-image, image edit/reference, and Image Omni multi-reference or series workflows via kling_official_image
• Text-to-speech via kling_tts when you already know the official Kling voice_id
• Cloud avatar presenter clips via kling_avatar, without replacing local talking_head
• Cloud lip-sync via kling_lip_sync, with explicit face selection for multi-person videos
• Accounts that need to use official Kling model permissions, resource packs, or regional endpoints

Notes

• provider="kling_official" is intentionally different from fal.ai's provider="kling".
• Official Kling is a paid remote API. OpenMontage uses conservative cost estimates and includes high-cost factors such as Omni references, series output, 4k mode, and native sound.
• Local image paths are sent as raw base64 for supported Classic/image-generation fields. Turbo image-to-video requires a URL and will not silently upload through fal.ai.
• Video Omni and Image Omni can pass official element_id references through element_list; Elements remain an internal Kling Official helper, not a standalone OpenMontage capability.
• Account Usage is available as a low-frequency diagnostic helper under tools/_kling/account.py; it is not a selector or pipeline tool.
• callback_url is passed through and recorded when supplied, but OpenMontage still polls tasks by default.
• kling_tts requires an explicit voice_id; OpenMontage does not guess a default official voice.
• kling_avatar and kling_lip_sync register under the existing avatar capability and coexist with local SadTalker/Wav2Lip tools. Current avatar pipelines must opt into them explicitly; registry discovery alone does not replace local tools.
• Official Kling audio effects and video effects are documented but intentionally not registered as OpenMontage tools yet, because current pipelines do not have a stable sound-effects or video-effects capability slot for them.

ElevenLabs — Voice, Music, Sound Effects Copy Section

Premium voice quality. Best TTS for narration-heavy videos. Also generates music and sound effects.

Tools unlocked: elevenlabs_tts, music_gen
Env var: ELEVENLABS_API_KEY

Setup

1. Go to elevenlabs.io and click Sign up
2. Go to Profile (bottom-left) > API Keys, or visit elevenlabs.io/app/settings/api-keys
3. Click Create API Key, name it, copy it
4. Add to .env: ELEVENLABS_API_KEY=xi_your-key-here

Pricing

Free tier: 10,000 characters/month (roughly 2-3 minutes of narration). API access included. Music generation and sound effects also available on free tier with limited credits.

fish.audio — Expressive TTS + Voice Cloning Copy Section

High-emotion narration and reusable cloned voices. S2-generation models support inline emotion tags ([laugh], [whispers]) and 80+ languages. Voices created in the fish.audio playground are reused across runs via reference_id.

Tools unlocked: fish_audio_tts
Env var: FISH_AUDIO_API_KEY

Setup

1. Sign up at fish.audio
2. Create an API key at fish.audio/go-api/api-keys
3. Add to .env: FISH_AUDIO_API_KEY=your-key-here
4. (Optional) Build or pick a voice model in the fish.audio playground and pass its id as reference_id to reuse a cloned voice

Backend models

model is required — there is no default. Pass one of:

s2.1-pro-free caveats — promotional, not a durable free tier. Per the fish.audio announcement, free API access runs through August 31, 2026 and is subject to Fair Use limits, carries no SLA or latency guarantee, requests may be retained by fish.audio, and commercial use is restricted. Don't route client work or production narration through it, and don't plan long-term costs at $0 — fish_audio_tts.estimate_cost() falls back to the paid s2.1-pro rate after the promotional window ends.

The legacy speech-1.x tier and s1-mini have been removed from the fish.audio API and are not supported.

Pricing

Billing is per UTF-8 byte of input text (not per character) — CJK text and emoji cost 3-4x an ASCII character of the same visible length. Current list pricing: s1 / s2-pro / s2.1-pro = $15 per 1M bytes; s2.1-pro-free is $0 only during the promotional window (through August 31, 2026 — see caveats above). Verify current pricing in the official pricing guide before large batches.

Doubao Speech — Mandarin TTS Copy Section

Strong Mandarin narration. Volcengine Doubao Speech is a good choice for Chinese explainer voiceovers and long-form narration that needs subtitle timing metadata.

Tools unlocked: doubao_tts
Env vars: DOUBAO_SPEECH_API_KEY, DOUBAO_SPEECH_VOICE_TYPE

Setup

1. Open the Volcengine Doubao Speech console and enable Speech Synthesis 2.0.
2. Create a new-console API Key.
3. Choose a Speech 2.0 voice type, for example zh_female_vv_uranus_bigtts.
4. Add to .env:
   bash Copy

   DOUBAO_SPEECH_API_KEY=your-api-key
   DOUBAO_SPEECH_VOICE_TYPE=zh_female_vv_uranus_bigtts

API Notes

OpenMontage uses the new-console API key flow:

X-Api-Key: ${DOUBAO_SPEECH_API_KEY}
X-Api-Resource-Id: seed-tts-2.0

Do not pass a new-console API Key as X-Api-App-Id or X-Api-Access-Key. That mismatch can produce load grant: requested grant not found.

What It Is Best For

• Natural Mandarin narration for Chinese-language explainers
• Async long-form narration via /api/v3/tts/submit and /api/v3/tts/query
• Character-level timing metadata for subtitle alignment
• Calm educational pacing where the video duration can follow the approved voice rhythm

Pacing

Start with speech_rate: 0 for natural Mandarin delivery. If the approved format needs a tighter runtime, compare short samples at speech_rate: 25 or 50 before generating the full narration. Do not force Doubao to match another provider's duration unless the user explicitly wants that tradeoff.

Pricing

Doubao Speech 2.0 is billed by character package or usage in Volcengine. OpenMontage estimates cost from text length and prefers provider-returned usage metadata when available.

Tencent Hunyuan Cloud — Video Generation Copy Section

Tencent Hunyuan (腾讯混元) cloud video generation via TokenHub API. Generates
videos from text or images using Tencent's Hunyuan models through the Tencent
TokenHub API — an OpenAI-compatible gateway (tokenhub.tencentmaas.com) with
simple Bearer-token authentication. No TC3-HMAC-SHA256 signing required.

Tools unlocked: hunyuan_cloud_video
Env var: TENCENT_TOKENHUB_API_KEY

Setup

1. Go to the Tencent Cloud TokenHub console.
2. Create an application or navigate to the API Key section.
3. Generate an API key and copy its value.
4. Add to .env:
   bash Copy

   TENCENT_TOKENHUB_API_KEY=your-tokenhub-api-key

What It's Best For

• Chinese-friendly prompt understanding — Hunyuan models natively understand Chinese prompts better than most Western APIs
• Simple auth — Bearer token, no complex signing (just an HTTP Authorization header)
• Direct Tencent Cloud quota — uses your own Tencent Cloud credits, not a third-party gateway mark-up
• Both T2V and I2V — one API key unlocks text-to-video and image-to-video

API Notes

TokenHub uses a submit-then-poll pattern:

# Submit a generation task
POST https://tokenhub.tencentmaas.com/v1/api/video/submit
Authorization: Bearer ${TENCENT_TOKENHUB_API_KEY}

# Poll for results
POST https://tokenhub.tencentmaas.com/v1/api/video/query
Authorization: Bearer ${TENCENT_TOKENHUB_API_KEY}

Resolution options: 720p (default) or 1080p.

A watermark (logo_add) is added by default. Set logo_add: 0 to disable it (requires console approval from Tencent).

Schema constraints:

• Prompt: max 200 UTF-8 characters
• Image: max 10MB, 50–5000 px per side, aspect ratio 1:4 to 4:1
• Formats: jpg, png, jpeg, webp, bmp, tiff

Fallback Tools

If hunyuan_cloud_video returns an error, the agent may retry with: jimeng_video, kling_official_video, minimax_video

Pricing

Tencent TokenHub uses a credit-based pricing system (1 credit = 1.2 RMB ≈ $0.167 USD):

Free tier: Tencent occasionally offers new-user credits for TokenHub. Check the TokenHub console for current promotions.

Azure AI Speech — Speech-to-Text Copy Section

Cloud transcription. Azure AI Speech Fast Transcription turns local audio into text with word-level timestamps, speaker diarization, and multi-language identification — no GPU required. Optional: the local faster-whisper transcriber remains the default offline STT path. When AZURE_SPEECH_KEY is set, the agent prefers azure_stt for cloud transcription.

Tools unlocked: azure_stt
Env vars: AZURE_SPEECH_KEY, AZURE_SPEECH_REGION (or AZURE_SPEECH_ENDPOINT)

Setup

1. In the Azure portal, create a Speech resource (Azure AI services → Speech service).
2. Open the resource's Keys and Endpoint page.
3. Copy KEY 1 and the Location/Region (e.g. eastus).
4. Add to .env:
   bash Copy

   AZURE_SPEECH_KEY=your-speech-resource-key
   AZURE_SPEECH_REGION=eastus
   # AZURE_SPEECH_ENDPOINT=https://<custom>...  # optional, overrides region

API Notes

OpenMontage uses the Fast Transcription REST endpoint, which accepts a local
audio file directly (multipart upload) and returns a synchronous result — no
Azure Blob storage, SAS URLs, or async job polling:

POST https://{region}.api.cognitive.microsoft.com/speechtotext/transcriptions:transcribe?api-version=2024-11-15
Ocp-Apim-Subscription-Key: ${AZURE_SPEECH_KEY}

For files longer than ~2 hours or bulk jobs, use Azure Batch Transcription instead (not wired into OpenMontage).

What It Is Best For

• Cloud transcription with word-level timestamps and no local GPU
• Multi-language auto-detection across a candidate locale set
• Speaker diarization without a HuggingFace token
• Subtitle timing metadata that flows straight into subtitle_gen

Pricing

Azure AI Speech Standard (S0) bills speech-to-text by audio-hour (roughly
$1.00/audio-hour at time of writing; a free F0 tier includes a limited monthly
allowance). OpenMontage estimates cost from the transcribed audio duration. See
Azure AI Speech pricing for current rates.

Azure AI Speech — Text-to-Speech Copy Section

Cloud neural narration. Azure neural TTS delivers high-quality multilingual voices with SSML prosody control and express-as styles — same Speech resource as azure_stt, so one key/region unlocks both directions. Optional: the local piper_tts remains the default offline TTS path. When AZURE_SPEECH_KEY is set, the agent may prefer azure_tts for cloud narration.

Tools unlocked: azure_tts
Env vars: AZURE_SPEECH_KEY, AZURE_SPEECH_REGION (or AZURE_TTS_ENDPOINT)

Setup

Identical to the STT setup above — the same Speech resource key and region work
for both. If you already configured azure_stt, azure_tts is available now.

AZURE_SPEECH_KEY=your-speech-resource-key
AZURE_SPEECH_REGION=eastus
# AZURE_TTS_ENDPOINT=https://<region>.tts.speech.microsoft.com  # optional, overrides region

Note: the TTS host (<region>.tts.speech.microsoft.com) differs from the STT
endpoint, so the optional override var is AZURE_TTS_ENDPOINT, not
AZURE_SPEECH_ENDPOINT.

API Notes

OpenMontage uses the synchronous REST v1 endpoint with an SSML body — no token
exchange, Blob storage, or job polling:

POST https://{region}.tts.speech.microsoft.com/cognitiveservices/v1
Ocp-Apim-Subscription-Key: ${AZURE_SPEECH_KEY}
Content-Type: application/ssml+xml
X-Microsoft-OutputFormat: audio-48khz-192kbitrate-mono-mp3

Voice shortlist aliases: andrew (default — warm, confident), brandon
(deeper), ava (bright female), guy (authoritative), jenny (friendly). Any
Azure voice short name is accepted verbatim. See the azure-text-to-speech
skill for SSML rate/pitch/style guidance.

What It Is Best For

• High-quality neural narration on existing Azure credentials
• Calm, confident explainer / founder-register delivery
• Multilingual narration via the Multilingual voice family
• Deterministic re-renders (fixed voice + SSML → identical audio)

Not for: fully offline production (use piper_tts) or voice cloning (use
elevenlabs_tts).

Pricing

Azure neural TTS Standard (S0) bills roughly $16 per 1M characters (a free
F0 tier includes a limited monthly allowance). A 150-word narration segment
costs about $0.015. OpenMontage estimates cost from character count. See
Azure AI Speech pricing for current rates.

Google — TTS + Imagen + Music + Video (Shared Key) Copy Section

One key, five tools. Google Cloud TTS has 700+ voices in 50+ languages — the strongest localization option. google_imagen supports both Imagen 4 and Gemini 2.5 Flash Image, including projects without Imagen catalog access. Google Lyria generates high-quality background music. Gemini Omni Flash supports conversational video editing, and direct Veo generation covers premium short video clips.

Tools unlocked: google_tts, google_imagen, google_music, gemini_omni_video, veo_video
Env var: GOOGLE_API_KEY (or GEMINI_API_KEY — either works; GEMINI_API_KEY takes precedence)

Setup

1. Go to Google AI Studio and sign in
2. Navigate to aistudio.google.com/apikey
3. Click Create API Key, select a Google Cloud project
4. Copy the key
5. Add to .env: GOOGLE_API_KEY=AIza... (or GEMINI_API_KEY=AIza...)

For TTS specifically, you also need to enable the Text-to-Speech API:

1. Visit console.cloud.google.com/apis/library/texttospeech.googleapis.com
2. Click Enable
3. Make sure your API key's restrictions allow the Text-to-Speech API

For Imagen, Lyria Music, Gemini Omni video, and direct Veo video, enable the Generative Language API:

1. Visit console.cloud.google.com/apis/library/generativelanguage.googleapis.com
2. Click Enable

Google TTS Pricing

The free tiers apply independently — you get 1M Standard AND 1M WaveNet AND 1M Neural2 characters per month free. That's roughly 250+ minutes of narration per month at zero cost.

Google Imagen Pricing

Free tier for Imagen: None. Paid tier only.

To select the Gemini backend through the governed image_selector, pass
preferred_provider: "google_imagen" and
model_name: "gemini-2.5-flash-image". The selector maps its neutral
model_name field to the provider's model input.

Gemini Omni Video Pricing

Generates 3–10 second clips at 720p/24fps with synthesized audio, plus stateful conversational editing (edit_video via previous_interaction_id). Paid tier only — no free tier. A typical 8-second clip costs ~$0.80; each edit turn generates a new clip and bills again.

Google Music (Lyria) Pricing

Free tier for Music: None. Paid tier only.

New account bonus: Google Cloud offers $300 in free credits for new accounts (90-day trial), applicable to TTS, Imagen, Music, Gemini Omni video, and direct Veo video.

Google TTS Voice Types

Google TTS offers 700+ voices across 50+ languages. Voice names follow the pattern {language}-{type}-{letter}:

Recommended voices: en-US-Chirp3-HD-Orus (male, rich/cinematic), en-US-Chirp3-HD-Aoede (female, warm). These are Google's newest tier — most natural-sounding, uses the v1beta1 endpoint automatically.

Languages include: English (US, UK, AU, IN), Spanish, French, German, Italian, Portuguese, Japanese, Korean, Chinese (Mandarin, Cantonese), Arabic, Hindi, Russian, Dutch, Polish, Turkish, Vietnamese, Thai, Indonesian, and 30+ more.

OpenAI — TTS + Image Generation Copy Section

Solid all-rounder. GPT Image 2 handles complex multi-element compositions and in-image text well. TTS is fast and affordable.

Tools unlocked: openai_tts, openai_image
Env var: OPENAI_API_KEY

Setup

1. Go to platform.openai.com/signup and create an account
2. Add a payment method at platform.openai.com/account/billing
3. Navigate to platform.openai.com/api-keys
4. Click Create new secret key, name it, copy it
5. Add to .env: OPENAI_API_KEY=sk-...

TTS Pricing

Image Pricing

Note: DALL-E 2/3 were shut down by OpenAI on 2026-05-12, and the gpt-image-1 family (gpt-image-1-mini, gpt-image-1.5) retires 2026-12-01 — gpt-image-2 is OpenAI's recommended replacement (deprecations).

Free tier: None. Requires prepaid billing. Previously offered $5 in free credits for new accounts (discontinued for most signups).

Runway — Native and Third-Party Video Models Copy Section

Multi-model production API. OpenMontage supports current Runway-native
models plus documented third-party Seedance 2.5, Gemini Omni Flash, and
MiniMax H3/Hailuo 3.0 routes.

Tools unlocked: runway_video
Env var: RUNWAY_API_KEY

Setup

1. Go to dev.runwayml.com and create a developer account
2. Subscribe to a paid plan (Standard or above — API requires subscription)
3. Generate an API key from the developer portal
4. Add to .env: RUNWAY_API_KEY=key_...

Pricing

API pricing (Runway credits are $0.01 each):

Seedance 2.5 supports text, image, and video inputs, 4–30 second outputs, and
up to 30 image, 10 video, and 10 audio references. Gemini Omni Flash supports
3–10 second text/image generation plus video editing with up to five image
references. Hailuo 3.0 is Runway's MiniMax H3 route and supports 5–15 second
outputs at 768P or 2K. The adapter maps each model to its exact request field
names instead of sending a generic payload.

Gen-3 Alpha Turbo and Gen-4 Aleph were removed from the Runway API on
2026-07-30 and are not offered by the tool.

Free tier: 125 one-time credits (no monthly renewal). Enough for about 5 seconds of Gen-4 video. API access requires a paid subscription.

Higgsfield — Multi-Model Video Orchestrator Copy Section

Multi-model video platform. Routes to Kling 3.0, Veo 3.1, Sora 2, WAN 2.5, and proprietary Soul Cinema through a single API. Includes Soul ID for character consistency across clips.

Tools unlocked: higgsfield_video
Env vars: HIGGSFIELD_API_KEY + HIGGSFIELD_API_SECRET (or combined HIGGSFIELD_KEY=key:secret)

Setup

1. Go to cloud.higgsfield.ai and create an account
2. Subscribe to a plan (Starter or above for API access)
3. Navigate to API Keys section at cloud.higgsfield.ai/api-keys
4. Generate an API key and secret
5. Add to .env:
   text Copy

   HIGGSFIELD_API_KEY=your-api-key
   HIGGSFIELD_API_SECRET=your-api-secret

Pricing

Per-generation costs (approximate, via credits):

Free tier: Limited credits on signup. No monthly renewal on free plan.

HeyGen — Avatar Video Gateway Copy Section

Multi-model video gateway. Access VEO, Sora, Runway, Kling, and Seedance through a single API.

Tools unlocked: heygen_video
Env var: HEYGEN_API_KEY

Setup

1. Go to app.heygen.com/register and create an account
2. Navigate to the API section in settings
3. Generate your API key
4. Add API balance (prepaid, separate from web plan credits)
5. Add to .env: HEYGEN_API_KEY=your-key-here

Pricing

Web plans:

Free tier: 1 credit on web platform. API is pay-as-you-go with prepaid balance.

Suno — AI Music Generation Copy Section

Full songs with vocals and lyrics. Any genre, up to 8 minutes. Instrumentals or vocal tracks.

Tools unlocked: suno_music
Env var: SUNO_API_KEY

Setup

1. Go to suno.com and create a Suno account
2. For API access, go to sunoapi.org and create an account
3. Navigate to the dashboard and copy your API key
4. Add credits (1 credit = $0.005 USD)
5. Add to .env: SUNO_API_KEY=your-key-here

Pricing

Suno platform:

API (via sunoapi.org): Pay-as-you-go, 1 credit = $0.005. Each generation produces 2 tracks.

Pexels — Free Stock Media Copy Section

Completely free. No cost, no attribution required, commercial use allowed.

Tools unlocked: pexels_image, pexels_video
Env var: PEXELS_API_KEY

Setup

1. Go to pexels.com/join and create a free account
2. Navigate to pexels.com/api
3. Click Your API Key or request API access
4. Copy your key from the dashboard
5. Add to .env: PEXELS_API_KEY=your-key-here

Pricing

Completely free. No paid tiers. No attribution required. Commercial use allowed.

• 200 requests/hour
• 20,000 requests/month
• Photo and video search + download

Pixabay — Free Stock Media Copy Section

Completely free. 5M+ royalty-free images and videos.

Tools unlocked: pixabay_image, pixabay_video
Env var: PIXABAY_API_KEY

Setup

1. Go to pixabay.com/accounts/register and create a free account
2. Navigate to pixabay.com/api/docs
3. Your API key is displayed at the top of the docs page (after login)
4. Copy the key
5. Add to .env: PIXABAY_API_KEY=your-key-here

Pricing

Completely free. No paid tiers. No attribution required. Commercial use allowed.

• ~100 requests/minute
• 5,000 requests/hour
• Photo and video search + download
• Standard API limited to 1280px images (full resolution requires editorial API)

Local Providers (Free, No API Key) Copy Chapter

These providers run entirely on your machine. No network, no API key, no cost. Some require a GPU.

Remotion — Programmatic Video Composition Copy Section

React-based video rendering. Turns still images into animated video with spring physics, animated text cards, stat cards, charts, and transitions. This is the key fallback when no video generation providers are configured — the agent generates images and Remotion animates them into professional-looking video.

Tool: video_compose (with operation="render" — auto-routes to Remotion when needed)
Runtime: CPU (Node.js required)
Env var: None

Setup

# Included in make setup, or install manually:
cd remotion-composer && npm install && cd ..

Requires Node.js 18+ and npx. The remotion-composer/ project is included in the repo.

What Remotion Renders

When Does Remotion Activate?

The video_compose tool's render operation auto-detects when Remotion is needed:

• Cuts contain still images (.png, .jpg, etc.)
• Cuts have type set to text_card, stat_card, chart, etc.
• Cuts specify animation or transition_in/transition_out

If Remotion is not installed, compositions fall back to FFmpeg Ken Burns pan-and-zoom — functional but less engaging.

Cost: Free. Always local.

HyperFrames - HTML/CSS/GSAP Video Composition Copy Section

GSAP-native local rendering. HyperFrames is the preferred runtime for motion-graphics-heavy HTML compositions and the character-animation pipeline's rigged SVG character acting.

Tool: hyperframes_compose directly, or video_compose with edit_decisions.render_runtime="hyperframes"
Runtime: CPU (Node.js >= 22, FFmpeg, and npx required)
Env var: None

Setup

node --version
ffmpeg -version
npx --yes hyperframes doctor

The CLI is consumed as npx hyperframes. Do not use npx @hyperframes/cli; that package name is not the OpenMontage runtime path.

What HyperFrames Renders

HyperFrames workspaces live under projects//hyperframes/. Final videos still follow the normal OpenMontage convention: projects//renders/final.mp4.

Cost: Free. Always local.

Piper TTS — Offline Text-to-Speech Copy Section

Completely free, fully offline TTS. No network required. Good quality for drafts and budget-constrained projects.

Tool: piper_tts
Runtime: CPU (no GPU needed)
Env var: None

Setup

# Install via pip
pip install piper-tts

# Or download the binary from GitHub
# https://github.com/rhasspy/piper/releases

# Download a voice model (first run downloads automatically)
piper --download-dir ~/.piper/models --model en_US-lessac-medium

Available voices: ~30 English voices plus voices for German, French, Spanish, Italian, and other languages. Lower variety than cloud providers but completely free and offline.

Quality: Good for drafts, internal videos, and budget projects. For client-facing narration, use ElevenLabs or Google TTS.

ComfyUI Video — Local Workflows and Hosted Partner Nodes Copy Section

Tool: comfyui_video

Optional env vars: COMFYUI_SERVER_URL (default
http://localhost:8188) and COMFYUI_VIDEO_SERVER_URL (video-specific
override).

The bundled WAN 2.2 workflows and caller-supplied local workflows execute on
the ComfyUI machine. MiniMax H3 is available as an official open-weight local
workflow; pass the official workflow exported in API format using
workflow_json or workflow_path, plus its output_node.

The MiniMax H3 local stack includes the pruned INT8 diffusion model, Qwen3-VL
text encoder, video VAE, and audio VAE. OpenMontage exposes the official
download URLs and destination folders in tool metadata rather than silently
downloading large weights.

The same tool also supports these ComfyUI Partner Nodes:

Partner Nodes are not offline: they require current ComfyUI, network access, a
logged-in Comfy account, and prepaid credits. Prices are estimates converted
from Comfy credits (211 credits = $1); actual metered usage is authoritative.

Official references: Partner Node overview,
pricing, and
MiniMax H3 local tutorial.

Local Video Generation (GPU Required) Copy Section

Free AI video generation. Requires an NVIDIA GPU with sufficient VRAM.

Tools: wan_video, hunyuan_video, cogvideo_video, ltx_video_local
Runtime: Local GPU (CUDA required)
Env vars: VIDEO_GEN_LOCAL_ENABLED=true, VIDEO_GEN_LOCAL_MODEL=<model>

Setup

# 1. Install the GPU stack
make install-gpu
# Or manually:
pip install diffusers transformers accelerate torch pillow requests

# 2. Enable local generation in .env
VIDEO_GEN_LOCAL_ENABLED=true

# 3. Choose a model based on your GPU VRAM
VIDEO_GEN_LOCAL_MODEL=wan2.1-1.3b      # 6GB+ VRAM (entry-level)
VIDEO_GEN_LOCAL_MODEL=wan2.1-14b       # 24GB+ VRAM (best local quality)
VIDEO_GEN_LOCAL_MODEL=hunyuan-1.5      # 12GB+ VRAM
VIDEO_GEN_LOCAL_MODEL=ltx2-local       # 8GB+ VRAM (fastest)
VIDEO_GEN_LOCAL_MODEL=cogvideo-5b      # 10GB+ VRAM
VIDEO_GEN_LOCAL_MODEL=cogvideo-2b      # 6GB+ VRAM (lightest)

Model Comparison

All local models support: Image-to-video, text-to-video, offline generation, seeded reproducibility.

Local Diffusion — Offline Image Generation (GPU Required) Copy Section

Free Stable Diffusion image generation. No API cost, fully offline.

Tool: local_diffusion
Runtime: Local GPU (CUDA required)
Env var: None (enable by installing dependencies)

Setup

pip install diffusers transformers accelerate torch

First run downloads the model (~4GB). Subsequent runs use the cached model.

VRAM requirement: 4GB+ (8GB recommended for 1024x1024 images)

Supports: Negative prompts, seeds, custom sizes. Quality is lower than FLUX or GPT Image 2 but completely free and offline.

LTX-2 on Modal — Self-Hosted Cloud GPU Copy Section

Run LTX-2 on Modal's cloud GPUs. Your own endpoint, your own scale. More consistent than local GPU, cheaper than commercial APIs.

Tool: ltx_video_modal
Runtime: Cloud (self-hosted)
Env var: MODAL_LTX2_ENDPOINT_URL

Setup

1. Create a Modal account
2. Deploy the LTX-2 endpoint (see Modal docs)
3. Set the endpoint URL in .env: MODAL_LTX2_ENDPOINT_URL=https://your-modal-endpoint

Modal pricing: ~$0.99/hour for A100 GPU time. Cost per video depends on generation time.

Other Local Tools (Always Available) Copy Section

These tools require only FFmpeg or Python packages — no GPU, no API key.

Provider-to-Tool Mapping Copy Chapter

Capability Coverage Copy Chapter

How many providers cover each capability:

FAQ Copy Chapter

Q: What's the absolute minimum I need to produce a video?
A: FFmpeg + Node.js (both free, local). FFmpeg handles video assembly, audio mixing, and subtitles. With Node.js, Remotion renders still images into animated video — so even without any video generation API, the agent generates images and Remotion turns them into professional-looking video with spring animations, text cards, and transitions. Add Piper TTS for free narration and Pexels/Pixabay for free stock footage.

Q: I don't have any video generation providers. Can I still make videos?
A: Yes. The agent generates still images (via any image provider — even free stock from Pexels/Pixabay) and Remotion composes them into animated video with spring physics transitions, text cards, stat cards, and charts. This is the default path for explainer and animation pipelines when no video gen is configured.

Q: What's one low-friction way to get AI-generated images and video?
A: fal.ai (FAL_KEY) is one pay-as-you-go option with broad single-key coverage. It unlocks FLUX images plus multiple video providers. No subscription — pay only for what you generate.

Q: I have a GPU. What can I run locally for free?
A: Set VIDEO_GEN_LOCAL_ENABLED=true and install diffusers. You get WAN 2.1, Hunyuan, CogVideo, and LTX video generation plus Stable Diffusion image generation — all free, all offline.

Q: Which TTS provider should I use?
A: For quality → ElevenLabs. For localization (50+ languages) → Google TTS. For budget → Google free tier (1M chars/month). For offline → Piper.

Q: Do I need all these providers?
A: No. Start with what you have. The selector pattern auto-routes to whatever's available. Missing a provider? The system falls through to the next one automatically.