Remotion Rule: Transcribe Captions

What it is

This is a Remotion skill rule for transcribing audio files to generate timed captions in programmatic video projects. It uses the @remotion/install-whisper-cpp package, which bundles Whisper.cpp for local speech-to-text transcription without sending audio to external APIs.

The rule is part of the official Remotion Agent Skill collection. It targets developers building React-based videos with Claude Code, Cursor, or OpenAI Codex who need automatic subtitles.

How it saves time or tokens

Manual caption creation is tedious and error-prone. This rule teaches AI coding assistants the exact sequence: install whisper-cpp, download a model, transcribe audio, and convert the output to Remotion's caption format. Without the rule, an assistant might suggest external transcription APIs or generate incorrect package imports. The rule ensures the assistant produces working transcription code on the first attempt.

How to use

1. Install the Remotion skills collection:

npx skills add remotion-dev/skills

1. Add the whisper-cpp package to your Remotion project:

npx remotion add @remotion/install-whisper-cpp

1. Create a Node.js script that downloads the model, transcribes audio, and outputs caption data.

Example

import path from 'path';
import {
  downloadWhisperModel,
  installWhisperCpp,
  transcribe,
  toCaptions,
} from '@remotion/install-whisper-cpp';

const whisperPath = path.join(process.cwd(), 'whisper.cpp');
await installWhisperCpp({ to: whisperPath });
await downloadWhisperModel({
  model: 'medium.en',
  folder: whisperPath,
});

const result = await transcribe({
  inputPath: 'src/audio/narration.wav',
  whisperPath,
  model: 'medium.en',
  tokenLevelTimestamps: true,
});

const captions = toCaptions({ transcription: result });
console.log(JSON.stringify(captions, null, 2));

Related on TokRepo

• Featured Workflows — Top-rated skills and workflows for AI-assisted development
• AI Tools for Video — Tools for video generation and editing with AI

This tool integrates with standard development workflows and requires minimal configuration to get started. It is available as open-source software with documentation and community support through the official repository. The project follows semantic versioning for stable releases.

For teams evaluating this tool, the key advantage is reducing manual work in repetitive tasks. The automation provided by the built-in features means less custom code to maintain and fewer integration points to manage. This translates directly to lower maintenance costs and faster iteration cycles.

Common pitfalls

• The medium.en model is English-only but faster; use medium or large for multilingual transcription at the cost of slower processing.
• Whisper.cpp compiles native code on first install, so your system needs a C++ toolchain (Xcode CLI tools on macOS, build-essential on Linux).
• Token-level timestamps (tokenLevelTimestamps: true) are required for word-by-word caption rendering; without them you only get segment-level timing.