Extraction can fail when an application treats browser execution and product data as the same layer. They are separate contracts with separate failure modes.
There are five distinct states between a URL and a record.
- HTTP responseThe origin returned bytes and a status.
- Rendered documentJavaScript ran and the expected page state appeared.
- Normalized contentNavigation, noise and repeated chrome were removed.
- Typed fieldsValues were converted and checked against the requested shape.
- Observable resultThe caller received timing, final URL, cache state and a request ID.
Request the representation your product consumes.
This request extracts a name and numeric price from the controlled product demonstration page. Inspect outputs.structured.valid and its validation errors before accepting the fields.
Create an API key or create an account before running the example. Replace your_api_key with the secret shown at creation.
export CRAWLVOLT_API_KEY="your_api_key"
curl https://www.crawlvolt.com/v1/scrape \
-H "Authorization: Bearer $CRAWLVOLT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.crawlvolt.com/examples/product-demo.html",
"formats": ["structured"],
"extract": {
"fields": {
"name": { "selector": "main h1", "required": true },
"price": { "selector": "[itemprop=price]", "type": "number", "required": true }
}
}
}'Follow the typed JSON guide to test missing required fields on the demonstration page. The extraction reference documents field types and validation. If content needs interaction, use explicit browser waits.
The response separates data from diagnostics. A missing optional field does not erase valid fields, and field-level errors remain inspectable.
Treat extraction as a product contract, not a prompt.
Prompts are useful when the shape is unknown. Product pipelines usually know the fields they need. Deterministic selectors, value conversion and explicit validation make that contract testable and cacheable.
The structured extractor does not call a language model. Presets and schemas resolve fields against the rendered document and return confidence plus validation diagnostics.
Review the whole path.
- Define what counts as a successful target response.
- Wait for a page condition, not an arbitrary long sleep.
- Keep raw, cleaned and typed outputs distinguishable.
- Return stable error codes and a request identity.
- Bill only the operation the customer can actually use.