Guide / Node.js

Turn a website into Markdown with Node.js.

Use native fetch to extract one public page, reject empty results and save a Markdown file. No npm package required.

By CrawlVoltUpdated 15 September 2026

Start with one URL.

You need Node.js 24, a terminal and a public page you own or have permission to collect. Run node --version to check your installation. This example uses Node's built-in fetch API.

Create a CrawlVolt API key. Give it a name for this project and copy the secret when it appears. It is shown once.

Open API keys Create an account

Already registered? Sign in and continue to API keys. Keep the secret in your local environment or credential store.

Authenticated requests use your account's credits. You can inspect a page in the Scrape playground before integrating it into your service.

Download, set your key, run.

Save page_to_markdown.mjs in an empty folder, then open a macOS or Linux terminal in that folder. Replace the key placeholder and URL below.

export CRAWLVOLT_API_KEY="your_api_key"
node page_to_markdown.mjs https://example.com page.md

The script requests the main content and writes page.md. Open that file in your editor. Check that it contains the page's useful text, rather than a navigation menu, an error message or a login screen. An existing output file is never overwritten; choose another filename for another run.

A small request with explicit checks.

The API key travels in the Authorization header. The request asks for outputs.markdown, uses a 60-second client timeout and checks the HTTP status before parsing the response. The helper functions are exported so the documentation collection example can reuse them.

import { randomUUID } from 'node:crypto';
import { writeFile } from 'node:fs/promises';
import { pathToFileURL } from 'node:url';

export function publicUrl(value) {
  const url = new URL(value);
  if (!['http:', 'https:'].includes(url.protocol) || url.username || url.password) {
    throw new Error('Use an HTTP(S) URL without embedded credentials.');
  }
  url.hash = '';
  return url;
}

export async function request(endpoint, payload, {
  key = process.env.CRAWLVOLT_API_KEY,
  operationId = randomUUID(),
  timeoutMs = 60_000,
  fetchImpl = fetch,
} = {}) {
  if (!key?.trim()) throw new Error('Set CRAWLVOLT_API_KEY first.');
  if (!['scrape', 'map', 'browse'].includes(endpoint)) throw new Error('Unknown endpoint.');
  publicUrl(payload.url);
  const response = await fetchImpl(`https://www.crawlvolt.com/v1/${endpoint}`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${key}`,
      'Content-Type': 'application/json',
      'Idempotency-Key': operationId,
    },
    body: JSON.stringify(payload),
    signal: AbortSignal.timeout(timeoutMs),
  });
  if (!response.ok) {
    const error = new Error(`CrawlVolt returned HTTP ${response.status}. Check Activity before retrying.`);
    error.status = response.status;
    throw error;
  }
  return response.json();
}

export function requireMarkdown(result) {
  const markdown = result?.outputs?.markdown;
  if (typeof markdown !== 'string' || !markdown.trim()) {
    throw new Error('No Markdown returned. Inspect the page and request in Activity.');
  }
  return markdown;
}

if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
  try {
    const [url = 'https://example.com', output = 'page.md'] = process.argv.slice(2);
    const result = await request('scrape', { url, formats: ['markdown'], only_main_content: true }, {
      operationId: process.env.CRAWLVOLT_OPERATION_ID || randomUUID(),
    });
    await writeFile(output, requireMarkdown(result), { encoding: 'utf8', flag: 'wx' });
    console.log(`Saved ${output}. Credits reported: ${result.credits_used ?? 'see Activity'}.`);
  } catch (error) {
    console.error(error.name === 'TimeoutError' ? 'Request timed out. Check Activity before retrying.' : error.message);
    process.exitCode = 1;
  }
}

The .mjs extension enables JavaScript modules. Keep this code in a server process or a local script; a browser bundle would expose the API key.

Know what failed before trying again.

401 or 403: check the key, its permissions and the target in Activity.
402: check your remaining credits and the available plans.
429: reduce concurrency and wait before submitting more requests.
Timeout or connection failure: the server may have completed the request. Inspect Activity before retrying.
Empty Markdown: open the source page and check whether content requires a click or a particular selector.

There are no automatic retries. If you need to repeat the same operation, set CRAWLVOLT_OPERATION_ID to a unique value before its first attempt and reuse it with the identical request. Use a new value for each new operation. Do not reuse one key across different pages.

Make the first result useful.

Check headings, links and code samples against the original page. A non-empty response is a transport check, not a quality score. Compare a small set of representative pages before building a larger job.

Next, collect a bounded documentation sample, prepare chunks with source URLs or handle content loaded after a click.