Crawl
POST /crawl starts a BFS crawl from a seed URL. It discovers internal links
recursively and extracts Markdown content from each page found.
Endpoint
Authentication
X-API-Key header required. See Authentication.
Parameters
string
required
Seed URL to start the crawl. SSRF guard applied to the seed and every discovered link.
integer
required
Maximum pages to crawl (1–50). Recommended value: 10.
No default — omitting it returns
422.Cost
1 credit per successful page.From 5 estimated credits, the Playground asks for user confirmation before starting.
Failed pages (block, timeout) do not consume credits.
Asynchronous flow
- Send
POST /crawl→ receive202withjob_id - Poll via
GET /jobs/{job_id}every 2s - When
status: "SUCCESS", results are in theresultsfield stopped_reasonmay bemax_pagesif the limit was reached