Video Processing & Editing

Expert in FFmpeg-based video editing, processing automation, and export optimization for modern content creation workflows.

When to Use

✅ Use for:

• Automated video editing pipelines (script-to-video)

• Cutting, trimming, concatenating clips

• Adding transitions, effects, overlays

• Audio mixing and normalization

• Subtitle/caption handling

• Export optimization for platforms

• Batch video processing

• Color grading and correction

❌ NOT for:

• Real-time video editing UI (use DaVinci Resolve/Premiere)

• 3D compositing (use After Effects/Blender)

• Motion graphics animation (use After Effects)

• Basic screen recording (use OBS)

Technology Selection

Video Editing Tools

Tool

Speed

Features

Use Case

FFmpeg

Very Fast

CLI automation

Production pipelines

MoviePy

Medium

Python API

Programmatic editing

PyAV

Fast

Low-level control

Custom processing

DaVinci Resolve

Slow

Full NLE

Manual editing

Decision tree:

Need automation? → FFmpeg

Need Python API? → MoviePy

Need frame-level control? → PyAV

Need manual editing? → DaVinci Resolve

Common Anti-Patterns

Anti-Pattern 1: Not Using Keyframe-Aligned Cuts

Novice thinking: "Just cut the video at any timestamp"

Problem: Causes artifacts, black frames, and playback issues.

Wrong approach:

# ❌ Cut at arbitrary timestamp (not keyframe-aligned)

ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 -c copy output.mp4

# Result: Black frames, artifacts, sync issues

Why wrong:

• Video codecs use keyframes (I-frames) every 2-10 seconds

• Non-keyframe cuts require re-encoding

• Using -c copy (stream copy) without keyframe alignment breaks playback

• GOP (Group of Pictures) structure depends on keyframes

Correct approach 1: Re-encode for precise cuts

# ✅ Re-encode for frame-accurate cutting

ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 \

  -c:v libx264 -crf 18 -preset medium \

  -c:a aac -b:a 192k \

  output.mp4

# Frame-accurate, but slower (re-encoding)

Correct approach 2: Keyframe-aligned stream copy

# ✅ Fast cutting with keyframe alignment

# Step 1: Find keyframes near cut points

ffprobe -select_streams v -show_frames -show_entries frame=pkt_pts_time,key_frame \

  -of csv input.mp4 | grep ",1$" | awk -F',' '{print $2}'

# Step 2: Cut at nearest keyframes (fast, no re-encoding)

ffmpeg -i input.mp4 -ss 00:01:22.000 -to 00:02:46.000 -c copy output.mp4

# Blazing fast, no quality loss, but not frame-accurate

Correct approach 3: Two-pass for best of both worlds

# ✅ Fast seek + precise cut

ffmpeg -ss 00:01:20.000 -i input.mp4 \

  -ss 00:00:03.456 -to 00:01:25.678 \

  -c:v libx264 -crf 18 -preset medium \

  -c:a aac -b:a 192k \

  output.mp4

# -ss BEFORE -i: Fast seek to keyframe (no decode)

# -ss AFTER -i: Precise trim (only decode needed portion)

Performance comparison:

Method

Time (1-hour video)

Accuracy

Quality

Stream copy (arbitrary)

2s

❌ Broken

❌ Artifacts

Stream copy (keyframe)

2s

±2s

✅ Perfect

Re-encode (simple)

15min

✅ Frame

⚠️ Quality loss

Two-pass (optimal)

3min

✅ Frame

✅ Perfect

Timeline context:

• 2010: FFmpeg required full re-encoding for cuts

• 2015: -c copy added for stream copying

• 2020: Two-pass cutting became best practice

• 2024: Hardware acceleration (NVENC) makes re-encoding viable

Anti-Pattern 2: Re-encoding Unnecessarily

Novice thinking: "Apply all edits in one FFmpeg command"

Problem: Multiple re-encodings cause cumulative quality loss.

Wrong approach:

# ❌ Re-encode for each operation (quality degradation)

# Operation 1: Trim

ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 \

  -c:v libx264 -crf 23 temp1.mp4

# Operation 2: Add audio

ffmpeg -i temp1.mp4 -i audio.mp3 -c:v libx264 -crf 23 \

  -map 0:v -map 1:a temp2.mp4

# Operation 3: Add subtitles

ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \

  -c:v libx264 -crf 23 output.mp4

# Result: 3x re-encoding = significant quality loss

Why wrong:

• Each re-encode is lossy (even with high CRF)

• Cumulative quality loss (generation loss)

• 3x encoding time

• Wasted disk I/O

Correct approach 1: Chain operations in single command

# ✅ Single-pass encoding with all operations

ffmpeg -ss 00:01:00 -i input.mp4 -i audio.mp3 \

  -to 00:04:00 \

  -vf "subtitles=subs.srt" \

  -map 0:v -map 1:a \

  -c:v libx264 -crf 18 -preset medium \

  -c:a aac -b:a 192k \

  output.mp4

# Single re-encode, all operations applied at once

Correct approach 2: Use stream copy when possible

# ✅ Lossless operations with stream copy

# Trim (stream copy)

ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 -c copy temp.mp4

# Add audio (stream copy video, encode audio)

ffmpeg -i temp.mp4 -i audio.mp3 \

  -map 0:v -map 1:a \

  -c:v copy -c:a aac -b:a 192k \

  temp2.mp4

# Burn subtitles (must re-encode video)

ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \

  -c:v libx264 -crf 18 -preset medium \

  -c:a copy \

  output.mp4

# Only 1 video re-encode (for subtitles)

Quality comparison:

Method

Encoding Passes

Quality (VMAF)

Time

3x re-encode (CRF 23)

3

82/100

45min

Single pass (CRF 23)

1

91/100

15min

Stream copy + 1 encode

1

95/100

18min

All stream copy

0

100/100

30s

Anti-Pattern 3: Ignoring Color Space Conversions

Novice thinking: "Just concatenate videos together"

Problem: Color shifts, mismatched brightness, broken playback.

Wrong approach:

# ❌ Concatenate videos with different color spaces

# clip1.mp4: BT.709 (HD), yuv420p

# clip2.mp4: BT.601 (SD), yuvj420p (full range)

# clip3.mp4: BT.2020 (HDR), yuv420p10le

# Create concat list

echo "file 'clip1.mp4'" > list.txt

echo "file 'clip2.mp4'" >> list.txt

echo "file 'clip3.mp4'" >> list.txt

# Concatenate without color normalization

ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

# Result: Color shifts between clips, broken HDR metadata

Why wrong:

• Different color spaces (BT.601 vs BT.709 vs BT.2020)

• Different pixel formats (yuv420p vs yuvj420p)

• Different color ranges (limited vs full)

• Metadata conflicts

Correct approach:

# ✅ Normalize color space before concatenation

# Step 1: Analyze color space of each clip

ffprobe -v error -select_streams v:0 \

  -show_entries stream=color_space,color_transfer,color_primaries,pix_fmt \

  -of default=noprint_wrappers=1 clip1.mp4

# Step 2: Normalize all clips to common color space

# Target: BT.709 (HD), yuv420p, limited range

# Normalize clip1 (already BT.709)

ffmpeg -i clip1.mp4 -c copy clip1_normalized.mp4

# Normalize clip2 (BT.601 SD → BT.709 HD)

ffmpeg -i clip2.mp4 \

  -vf "scale=in_range=full:out_range=limited,colorspace=bt709:iall=bt601:fast=1" \

  -color_primaries bt709 \

  -color_trc bt709 \

  -colorspace bt709 \

  -c:v libx264 -crf 18 -preset medium \

  -c:a copy \

  clip2_normalized.mp4

# Normalize clip3 (BT.2020 HDR → BT.709 SDR)

ffmpeg -i clip3.mp4 \

  -vf "zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=limited,format=yuv420p" \

  -color_primaries bt709 \

  -color_trc bt709 \

  -colorspace bt709 \

  -c:v libx264 -crf 18 -preset medium \

  -c:a copy \

  clip3_normalized.mp4

# Step 3: Concatenate normalized clips

echo "file 'clip1_normalized.mp4'" > list.txt

echo "file 'clip2_normalized.mp4'" >> list.txt

echo "file 'clip3_normalized.mp4'" >> list.txt

ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

Color space guide:

Standard

Color Space

Transfer

Primaries

Use Case

BT.601

SD

bt470bg

bt470bg

Old SD content

BT.709

HD

bt709

bt709

Modern HD/FHD

BT.2020

UHD/HDR

smpte2084

bt2020

4K HDR

sRGB

Web

iec61966-2-1

bt709

Web delivery

Anti-Pattern 4: Poor Audio Sync

Novice thinking: "Video and audio are separate, just overlay them"

Problem: Lip sync issues, audio drift, broken playback.

Wrong approach:

# ❌ Replace audio without sync consideration

ffmpeg -i video.mp4 -i audio.mp3 \

  -map 0:v -map 1:a \

  -c:v copy -c:a copy \

  output.mp4

# Problems:

# - Audio duration ≠ video duration

# - No audio stretching/compression

# - Drift over time

Why wrong:

• Audio and video have different durations

• No timebase synchronization

• No drift correction

• Ignores original audio sync

Correct approach 1: Stretch/compress audio to match video

# ✅ Adjust audio speed to match video duration

# Get durations

VIDEO_DUR=$(ffprobe -v error -show_entries format=duration \

  -of default=noprint_wrappers=1:nokey=1 video.mp4)

AUDIO_DUR=$(ffprobe -v error -show_entries format=duration \

  -of default=noprint_wrappers=1:nokey=1 audio.mp3)

# Calculate speed ratio

RATIO=$(echo "$VIDEO_DUR / $AUDIO_DUR" | bc -l)

# Stretch audio to match video (with pitch correction)

ffmpeg -i video.mp4 -i audio.mp3 \

  -filter_complex "[1:a]atempo=${RATIO}[a]" \

  -map 0:v -map "[a]" \

  -c:v copy -c:a aac -b:a 192k \

  output.mp4

Correct approach 2: Precise offset and trim

# ✅ Sync audio with offset and trim

# Audio starts 0.5s late, trim to match video

ffmpeg -i video.mp4 -itsoffset 0.5 -i audio.mp3 \

  -map 0:v -map 1:a \

  -shortest \

  -c:v copy -c:a aac -b:a 192k \

  output.mp4

# -itsoffset: Delay audio by 0.5s

# -shortest: Trim to shortest stream

Correct approach 3: Mix multiple audio tracks with sync

# ✅ Mix dialogue, music, effects with precise timing

ffmpeg -i video.mp4 -i dialogue.wav -i music.mp3 -i sfx.wav \

  -filter_complex "

    [1:a]adelay=0|0[dlg];

    [2:a]volume=0.3,adelay=500|500[mus];

    [3:a]adelay=1200|1200[sfx];

    [dlg][mus][sfx]amix=inputs=3:duration=first[a]

  " \

  -map 0:v -map "[a]" \

  -c:v copy -c:a aac -b:a 256k \

  output.mp4

# adelay: Precise millisecond timing

# amix: Mix multiple audio streams

# volume: Normalize levels

Audio sync checklist:

□ Verify video and audio durations match

□ Use -shortest to prevent excess audio

□ Apply adelay for precise timing offsets

□ Use atempo for speed adjustment (maintains pitch)

□ Set audio bitrate appropriately (128k-256k)

□ Test lip sync at beginning, middle, end

Anti-Pattern 5: Wrong Codec/Bitrate for Platform

Novice thinking: "One export settings for everything"

Problem: Wasted bandwidth, poor quality, rejected uploads, compatibility issues.

Wrong approach:

# ❌ Export everything at 4K 50 Mbps

ffmpeg -i input.mp4 \

  -c:v libx264 -b:v 50M -s 3840x2160 \

  -c:a aac -b:a 320k \

  output.mp4

# For Instagram story: 2 GB file, rejected (max 100 MB)

# For YouTube: Could use 10 Mbps and look identical

# For Twitter: Exceeds bitrate limits

Why wrong:

• Platform-specific size/bitrate limits

• Over-encoding wastes bandwidth

• Wrong resolution for platform

• Incompatible codecs

Correct approach: Platform-optimized exports

YouTube (recommended settings):

# ✅ YouTube 1080p upload

ffmpeg -i input.mp4 \

  -c:v libx264 -preset slow -crf 18 \

  -s 1920x1080 -r 30 \

  -pix_fmt yuv420p \

  -color_primaries bt709 -color_trc bt709 -colorspace bt709 \

  -movflags +faststart \

  -c:a aac -b:a 192k -ar 48000 \

  youtube_1080p.mp4

# YouTube 4K upload

ffmpeg -i input.mp4 \

  -c:v libx264 -preset slow -crf 18 \

  -s 3840x2160 -r 60 \

  -pix_fmt yuv420p \

  -movflags +faststart \

  -c:a aac -b:a 256k -ar 48000 \

  youtube_4k.mp4

Instagram (Stories, Reels, Feed):

# ✅ Instagram Story (9:16, max 100 MB, 15s)

ffmpeg -i input.mp4 \

  -c:v libx264 -preset medium -crf 23 \

  -s 1080x1920 -r 30 -t 15 \

  -pix_fmt yuv420p \

  -movflags +faststart \

  -c:a aac -b:a 128k \

  instagram_story.mp4

# ✅ Instagram Reel (9:16, max 90s)

ffmpeg -i input.mp4 \

  -c:v libx264 -preset medium -crf 23 \

  -s 1080x1920 -r 30 -t 90 \

  -pix_fmt yuv420p \

  -movflags +faststart \

  -c:a aac -b:a 128k \

  instagram_reel.mp4

# ✅ Instagram Feed (1:1 or 4:5)

ffmpeg -i input.mp4 \

  -c:v libx264 -preset medium -crf 23 \

  -s 1080x1080 -r 30 \

  -pix_fmt yuv420p \

  -movflags +faststart \

  -c:a aac -b:a 128k \

  instagram_feed.mp4

Twitter/X:

# ✅ Twitter video (max 512 MB, 2:20)

ffmpeg -i input.mp4 \

  -c:v libx264 -preset medium -crf 23 \

  -s 1280x720 -r 30 -t 140 \

  -maxrate 5000k -bufsize 10000k \

  -pix_fmt yuv420p \

  -movflags +faststart \

  -c:a aac -b:a 128k \

  twitter.mp4

TikTok:

# ✅ TikTok (9:16, max 287 MB, 10 min)

ffmpeg -i input.mp4 \

  -c:v libx264 -preset medium -crf 23 \

  -s 1080x1920 -r 30 -t 600 \

  -pix_fmt yuv420p \

  -movflags +faststart \

  -c:a aac -b:a 128k \

  tiktok.mp4

Web (HTML5 video):

# ✅ Web optimized (fast load, broad compatibility)

ffmpeg -i input.mp4 \

  -c:v libx264 -preset medium -crf 23 \

  -s 1920x1080 -r 30 \

  -pix_fmt yuv420p \

  -profile:v baseline -level 3.0 \

  -movflags +faststart \

  -c:a aac -b:a 128k -ar 48000 \

  web.mp4

Platform specs table:

Platform

Max Size

Max Duration

Resolution

FPS

Bitrate

Codec

YouTube

Unlimited

Unlimited

8K

60

Auto

H.264/VP9

Instagram Story

100 MB

15s

1080x1920

30

~5 Mbps

H.264

Instagram Reel

1 GB

90s

1080x1920

30

~8 Mbps

H.264

Twitter

512 MB

2:20

1920x1080

60

5 Mbps

H.264

TikTok

287 MB

10min

1080x1920

30

~4 Mbps

H.264

LinkedIn

5 GB

10min

1920x1080

30

5 Mbps

H.264

Web

Varies

Varies

1920x1080

30

2-5 Mbps

H.264

Export optimization checklist:

□ Use -movflags +faststart for web (progressive download)

□ Use -pix_fmt yuv420p for broad compatibility

□ Set -r 30 for most platforms (avoid variable framerate)

□ Use -preset slow for final exports (better quality)

□ Use -preset ultrafast for drafts

□ Apply -maxrate and -bufsize for streaming

□ Test playback on target platform before bulk export

Production Checklist

□ Align cuts to keyframes (or two-pass seek)

□ Chain operations in single FFmpeg command

□ Normalize color spaces before concatenating

□ Verify audio/video sync (test at multiple points)

□ Use platform-specific export presets

□ Apply -movflags +faststart for web delivery

□ Set proper color metadata (bt709 for HD)

□ Test output file on target platform

□ Keep lossless intermediate files (ProRes, FFV1)

□ Use hardware acceleration for batch jobs (NVENC, VideoToolbox)

When to Use vs Avoid

Scenario

Appropriate?

Automated video pipeline (script → video)

✅ Yes - FFmpeg automation

Batch process 100 videos

✅ Yes - parallel FFmpeg jobs

Trim/cut clips programmatically

✅ Yes - precise cutting

Add subtitles to videos

✅ Yes - burn or soft subs

Color grade footage

⚠️ Limited - basic only

Multi-cam editing

❌ No - use DaVinci Resolve

Motion graphics

❌ No - use After Effects

Real-time preview editing

❌ No - use Premiere/Resolve

References

• /references/ffmpeg-guide.md - Complete FFmpeg command reference

• /references/timeline-editing.md - Timeline concepts, multi-track editing

• /references/export-optimization.md - Platform-specific export settings

Scripts

• scripts/video_editor.py - Cut, trim, concatenate, transitions, effects

• scripts/batch_processor.py - Parallel batch video processing

This skill guides: Video editing | FFmpeg | Timeline editing | Transitions | Export optimization | Audio mixing | Color grading | Automated video production