FFmpeg — The Universal Multimedia Processing Toolkit

What it is

FFmpeg is the most widely used multimedia processing framework. It can decode, encode, transcode, mux, demux, stream, filter, and play almost every audio and video format ever created. Nearly every video application -- from VLC to YouTube's backend -- relies on FFmpeg's libraries or command-line tools under the hood.

The target audience spans video engineers, content creators automating batch processing, DevOps teams building media pipelines, and AI developers preparing training data from video sources.

How it saves time or tokens

FFmpeg replaces entire categories of commercial software with a single command-line binary. A video format conversion that requires a GUI tool and manual clicks becomes a one-liner. Batch processing hundreds of files uses a shell loop around a single FFmpeg command.

For AI workflows, FFmpeg is essential for extracting frames from video datasets, converting audio formats for speech models, and preparing media assets for training pipelines. Its filter graph system handles complex transformations (resize, crop, overlay, color correction) without writing code.

How to use

1. Install FFmpeg:

# macOS
brew install ffmpeg

# Ubuntu/Debian
sudo apt install ffmpeg

# Verify installation
ffmpeg -version

1. Convert a video format:

ffmpeg -i input.mov -c:v libx264 -crf 23 -c:a aac output.mp4

1. Extract audio from video:

ffmpeg -i video.mp4 -vn -acodec libmp3lame -q:a 2 audio.mp3

Example

Extract one frame per second from a video for AI training data:

# Extract frames at 1fps, save as numbered PNGs
ffmpeg -i input.mp4 -vf fps=1 frames/frame_%04d.png

# Resize and extract frames in one pass
ffmpeg -i input.mp4 -vf 'fps=1,scale=224:224' frames/frame_%04d.png

# Trim to first 60 seconds, then extract
ffmpeg -i input.mp4 -t 60 -vf fps=1 frames/frame_%04d.png

Related on TokRepo

• AI tools for video -- Video generation and processing tools
• Automation tools -- CLI tools for batch media workflows

Common pitfalls

• The -crf value controls quality vs. file size for x264/x265. Lower means higher quality. The default 23 is a reasonable starting point; going below 18 rarely produces visible improvement but doubles file size.
• Forgetting -c copy when you only want to trim or remux wastes time re-encoding the entire stream.
• Filter graph syntax uses commas for chaining filters and semicolons for separate filter chains. Mixing them up produces cryptic errors.
• Hardware acceleration (NVENC, VideoToolbox, VAAPI) requires building FFmpeg with the right flags. The default Homebrew/apt packages may not include all accelerators.
• Piping FFmpeg output to another process requires -f to specify the output format explicitly, since FFmpeg cannot infer format from a pipe.