MLX Transcribe Tools

MLX Transcribe: A tool for transcribing audio files using MLX Whisper.

Run this example on macOS or Linux with a supported MLX backend. Windows is not supported. Linux requires one of mlx[cpu], mlx[cuda12], or mlx[cuda13]. See MLX installation.

mlx_transcribe_tools.py

"""
MLX Transcribe: A tool for transcribing audio files using MLX Whisper

Requirements:
1. ffmpeg - Install using:
   - macOS: `brew install ffmpeg`
   - Ubuntu: `sudo apt-get install ffmpeg`
   - Windows: Download from https://ffmpeg.org/download.html

2. mlx-whisper library:
   uv pip install mlx-whisper

Example Usage:
- Place your audio files in the 'storage/audio' directory
    Eg: download https://www.ted.com/talks/reid_hoffman_and_kevin_scott_the_evolution_of_ai_and_how_it_will_impact_human_creativity
- Run this script to transcribe audio files
- Supports various audio formats (mp3, mp4, wav, etc.)
"""

from pathlib import Path

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.mlx_transcribe import MLXTranscribeTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------


# Get audio files from storage/audio directory
agno_root_dir = Path(__file__).parent.parent.parent.resolve()
audio_storage_dir = agno_root_dir.joinpath("storage/audio")

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    if not audio_storage_dir.exists():
        audio_storage_dir.mkdir(exist_ok=True, parents=True)

    agent = Agent(
        name="Transcription Agent",
        model=OpenAIChat(id="gpt-4o"),
        tools=[MLXTranscribeTools(base_dir=audio_storage_dir)],
        instructions=[
            "To transcribe an audio file, use the `transcribe` tool with the name of the audio file as the argument.",
            "You can find all available audio files using the `read_files` tool.",
        ],
        markdown=True,
    )

    agent.print_response(
        "Summarize the reid hoffman ted talk, split into sections", stream=True
    )

Run the Example

1. Set up your virtual environment

   Mac

   uv venv --python 3.12
   source .venv/bin/activate
   

   uv venv --python 3.12
   .venv\Scripts\activate
   
2. Install dependencies

   uv pip install -U agno mlx-whisper openai
   
3. Export your OpenAI API key

   Mac/Linux

   export OPENAI_API_KEY="your_openai_api_key_here"
   

   $Env:OPENAI_API_KEY="your_openai_api_key_here"
   
4. Install an MLX backend

   On macOS, the dependency step installs the standard MLX package. On Linux, install exactly one backend for your hardware before running: uv pip install -U "mlx[cpu]", uv pip install -U "mlx[cuda12]", or uv pip install -U "mlx[cuda13]".

5. Install ffmpeg

   Install ffmpeg using the macOS or Ubuntu command in the source header.

6. Add an audio file

   For a standalone file, replace the agno_root_dir assignment with agno_root_dir = Path(__file__).parent.resolve(). Save the code as mlx_transcribe_tools.py, then add the audio file to storage/audio beside the script.

7. Run the example

   Save the code above as mlx_transcribe_tools.py, then run:

   python mlx_transcribe_tools.py
   

Full source: cookbook/91_tools/mlx_transcribe_tools.py