SDK, CLI and MCP

Connect CrawlVolt to Python, TypeScript, command-line tools and applications that use MCP.

CrawlVolt exposes Search, Scrape, Browse, Map, Crawl, and Usage through REST, SDK sources, a CLI, and Model Context Protocol tools. Product calls preserve the API request_id so failures can be matched with Activity logs.

Hosted MCP

The recommended setup connects directly to https://www.crawlvolt.com/v1/mcp. It needs no local package or background process. Create a project API key in the CrawlVolt console, then expose it to your MCP client as CRAWLVOLT_API_KEY.

Codex

Add this to ~/.codex/config.toml or a trusted project's .codex/config.toml:

[mcp_servers.crawlvolt]
url = "https://www.crawlvolt.com/v1/mcp"
bearer_token_env_var = "CRAWLVOLT_API_KEY"
tool_timeout_sec = 120

Codex CLI, the IDE extension, and the desktop app share this configuration. Run /mcp or codex mcp list to check the connection.

Claude Code

Add this project-scoped .mcp.json:

{
  "mcpServers": {
    "crawlvolt": {
      "type": "http",
      "url": "https://www.crawlvolt.com/v1/mcp",
      "headers": {
        "Authorization": "Bearer ${CRAWLVOLT_API_KEY}"
      }
    }
  }
}

Claude Code asks for approval before using a project-scoped server. Use /mcp or claude mcp list to inspect its status.

Verify the endpoint

curl --request POST https://www.crawlvolt.com/v1/mcp \
  --header "Authorization: Bearer $CRAWLVOLT_API_KEY" \
  --header "Content-Type: application/json" \
  --header "Accept: application/json, text/event-stream" \
  --data '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

The endpoint exposes six deterministic tools:

crawlvolt_search
crawlvolt_scrape
crawlvolt_browse
crawlvolt_map
crawlvolt_crawl
crawlvolt_get_usage

Tool calls consume the same credits and enforce the same API-key scopes as the corresponding REST endpoint. Results include structuredContent, a text form for the model, and isError for product-level failures.

Local MCP and CLI

The Python source under integrations/python provides the crawlvolt and crawlvolt-mcp entry points. Use the local stdio server when a client cannot connect to Streamable HTTP. crawlvolt init creates guarded examples for Codex, Claude, generic MCP clients, and the local stdio fallback without overwriting existing files.

crawlvolt init .
crawlvolt search "browser automation observability" --limit 5
crawlvolt map https://docs.example.com --body '{"max_pages":20}'
crawlvolt logs --issues

The Python package is validated from source; PyPI publication is still pending. Until then, install it from a repository checkout rather than relying on an unowned registry name.

TypeScript

import { CrawlVolt } from '@crawlvolt/sdk';

const client = new CrawlVolt({ apiKey: process.env.CRAWLVOLT_API_KEY! });
const result = await client.search({ query: 'browser automation observability', limit: 5 });
console.log(result.request_id, result.data.web);

The client creates idempotency keys for POST calls, retries only transient failures, and raises CrawlVoltError with status, code, retryability, and request ID. The package is validated from source; npm publication is pending.

Python

import os
from crawlvolt import AsyncCrawlVolt

client = AsyncCrawlVolt(os.environ["CRAWLVOLT_API_KEY"])
result = await client.scrape(url="https://example.com", formats=["markdown", "metadata"])
print(result["request_id"], result["outputs"]["markdown"])

Automation and agents

The repository includes an importable n8n Search-to-Scrape workflow, Pipedream Search/Scrape actions, Claude and Codex MCP examples, and thin Python adapters for LangChain, LlamaIndex, and CrewAI. They delegate authentication, retries, and errors to the API clients instead of duplicating transport behavior.