Video Caption Agent

Extract audio, transcribe it to timed SRT captions with OpenAI, and embed captions with MoviePyVideoTools.

Create an agent that extracts audio from a video, generates timestamped SRT captions, and embeds them into the output video.

video_caption.py

from pathlib import Path
from sys import argv

from agno.agent import Agent
from agno.models.openai import OpenAIResponses
from agno.tools.moviepy_video import MoviePyVideoTools
from openai import OpenAI

video_tools = MoviePyVideoTools(
    enable_process_video=True,
    enable_generate_captions=False,
    enable_embed_captions=True,
)


def transcribe_audio_to_srt(audio_path: str, output_path: str) -> str:
    """Transcribe an audio file to a timestamped SRT file."""
    with open(audio_path, "rb") as audio_file:
        captions = OpenAI().audio.transcriptions.create(
            model="whisper-1",
            file=audio_file,
            response_format="srt",
        )

    Path(output_path).write_text(captions, encoding="utf-8")
    return output_path


video_caption_agent = Agent(
    name="Video Caption Generator Agent",
    model=OpenAIResponses(id="gpt-5.2"),
    tools=[video_tools, transcribe_audio_to_srt],
    instructions=[
        "Follow the requested file paths exactly.",
        "Call extract_audio, then transcribe_audio_to_srt, then embed_captions.",
        "Return the captioned video path.",
    ],
)


if __name__ == "__main__":
    if len(argv) != 2:
        raise SystemExit("Usage: python video_caption.py /path/to/input.mp4")

    video_path = Path(argv[1]).expanduser().resolve()
    if not video_path.is_file():
        raise SystemExit(f"Video not found: {video_path}")

    audio_path = video_path.with_suffix(".wav")
    srt_path = video_path.with_suffix(".srt")
    output_path = video_path.with_name(f"{video_path.stem}_captioned.mp4")

    video_caption_agent.print_response(
        f"""Process this video using the exact paths below.
Video: {video_path}
Audio: {audio_path}
SRT: {srt_path}
Captioned video: {output_path}"""
    )

Agno v2.7.2 uses Arial for embedded caption text. Install Arial before running this example on a system that does not include it.

Run the Example

1. Set up your virtual environment

   Mac

   uv venv --python 3.12
   source .venv/bin/activate
   

   uv venv --python 3.12
   .venv\Scripts\activate
   
2. Install dependencies

   uv pip install -U agno moviepy openai
   
3. Install FFmpeg

   Install FFmpeg, then verify the executable:

   ffmpeg -version
   
4. Export your OpenAI API key

   export OPENAI_API_KEY="your_openai_api_key"
   
5. Run the agent

   python video_caption.py /absolute/path/to/input.mp4