Convert a web page to Markdown with Python.

Start with one page.

Store the key in CRAWLVOLT_API_KEY using the terminal command below. Keep it out of source control and shared files.

You can inspect a page in the Scrape playground first. The anonymous demo is limited; authenticated requests use credits from your account. Choose a public page you own or have permission to collect.

Request Markdown with Python.

Save this as page_to_markdown.py. It uses the Python standard library and makes one HTTP request.

import json
import os
import sys
import uuid
from urllib.request import Request, urlopen

url = sys.argv[1] if len(sys.argv) > 1 else "https://example.com"
operation_id = os.environ.get("CRAWLVOLT_OPERATION_ID") or str(uuid.uuid4())
request = Request(
    "https://www.crawlvolt.com/v1/scrape",
    data=json.dumps({"url": url, "formats": ["markdown"]}).encode("utf-8"),
    headers={
        "Authorization": "Bearer " + os.environ["CRAWLVOLT_API_KEY"],
        "Content-Type": "application/json",
        "Idempotency-Key": operation_id,
    },
    method="POST",
)
with urlopen(request, timeout=60) as response:
    result = json.load(response)

markdown = result.get("outputs", {}).get("markdown")
if not isinstance(markdown, str) or not markdown.strip():
    raise ValueError("No Markdown returned; inspect the request in Activity.")
print(markdown)

Download the Python example

export CRAWLVOLT_API_KEY="your_api_key"
python3 page_to_markdown.py https://example.com

In a macOS or Linux terminal, replace your_api_key with your own key. The script prints the returned Markdown. Inspect the output for your chosen page before connecting it to your application.

Check the content your application needs.

Verify the title, main text, links and any tables you rely on. A successful request alone does not prove that every required field is present. Activity shows request details and credit usage.

The example does not retry automatically. To retry one operation, set CRAWLVOLT_OPERATION_ID to a unique value before the first attempt and reuse it for that attempt only. Use a new value for each new operation.

Connect it to your workflow.

Once the output is useful, send the Markdown to your existing storage or indexing step. This request extracts one page; scheduling, chunking and embeddings are separate steps to implement and verify.

For recurring use, estimate pages multiplied by refresh frequency and measure actual credits on a small sample. Compare the current plans against the operations you need.

Try a page or continue with the n8n example.