Firecrawl

Use Firecrawl with Agno to scrape and crawl the web.

FirecrawlTools enable an Agent to perform web crawling and scraping tasks.

Prerequisites

The following example requires the firecrawl-py library and an API key which can be obtained from Firecrawl.

uv pip install -U firecrawl-py openai

export FIRECRAWL_API_KEY=***

Example

The following agent will crawl https://finance.yahoo.com/ and return a summary of the content:

from agno.agent import Agent
from agno.tools.firecrawl import FirecrawlTools

agent = Agent(tools=[FirecrawlTools(enable_scrape=False, enable_crawl=True)], markdown=True)
agent.print_response("Summarize this https://finance.yahoo.com/")

Toolkit Params

Parameter	Type	Default	Description
api_key	str	None	API key for authentication. Uses FIRECRAWL_API_KEY env var if not provided
enable_scrape	bool	True	Enables website scraping functionality
enable_crawl	bool	False	Enables website crawling functionality
enable_mapping	bool	False	Enables website mapping functionality
enable_search	bool	False	Enables web search functionality
all	bool	False	Enables all functionality when set to True
formats	List[str]	None	List of formats to be used for operations
limit	int	10	Maximum number of items to retrieve
poll_interval	int	30	Interval in seconds between polling for results
search_params	Dict[str, Any]	None	Parameters for search operations
api_url	str	https://api.firecrawl.dev	API URL to use for the Firecrawl app

Toolkit Functions

Function	Description
scrape_website	Scrapes a website using Firecrawl. Parameters include url to specify the URL to scrape. The function supports optional formats if specified. Returns the results of the scraping in JSON format.
crawl_website	Crawls a website using Firecrawl. Parameters include url to specify the URL to crawl. The maximum number of pages crawled is set by the toolkit-level limit parameter. The function supports optional formats and returns the crawling results in JSON format.
map_website	Maps a website structure using Firecrawl. Parameters include url to specify the URL to map. Returns the mapping results in JSON format.
search_web	Performs a web search using Firecrawl. Parameters include query for the search term. The maximum number of results is set by the toolkit-level limit parameter. Returns search results in JSON format.

Developer Resources

• Tools