Proxies and difficult targets
CrawlVolt is a browser execution and web access layer, not an anti-bot bypass service. JavaScript rendering, stable extraction and controlled interaction are product features; defeating a website's access controls is not.
Public support levels
| Target behavior | V2 support |
|---|---|
| Public page that needs JavaScript | Supported |
| Customer-owned authenticated account | Supported through a short-lived CrawlVolt session |
| Transient DNS, connection or browser-engine failure | Retryable with bounded client backoff |
Upstream 403 or 429 | Returned as target state; best-effort only |
| CAPTCHA or access challenge | Unsupported; no public solving action |
| Device fingerprint or identity-verification bypass | Unsupported |
| Customer-supplied arbitrary proxy | Unsupported |
CrawlVolt does not promise universal website access. A successful browser run can still report an upstream denial in status_code; clients must treat that as target behavior, not extracted content success.
V2 proxy decision
The preview uses platform-controlled direct egress. It does not accept a proxy URL in the public request because arbitrary proxies create SSRF, credential-exfiltration, attribution and abuse risks.
A future managed-egress option may provide region selection and reliability for legitimate workloads. That capability must use allowlisted provider integrations, project-isolated credentials encrypted at rest, destination policy enforcement after every redirect, and separate cost metering. Rotation would be a reliability mechanism, not a promise to evade blocks.
Residential proxy networks and CAPTCHA-solving marketplaces are not part of the core CrawlVolt offer. Any future provider evaluation requires a separate legal, abuse and unit-economics review.
Retry behavior
CrawlVolt does not hide repeated target attempts inside one billed request today. Clients may retry typed 429, 502, 503 and 504 responses with exponential backoff, jitter and a stable Idempotency-Key. Do not retry authentication, validation, SSRF-policy or upstream access-denial responses automatically.
Future internal retries must remain bounded, expose the attempt count and never multiply billing units for one idempotent operation.
Required architecture before managed egress
- Dedicated browser egress network with private, loopback, link-local and metadata ranges denied after DNS resolution and redirects.
- Per-project proxy credentials and concurrency limits, never credentials supplied inline by a client.
- Provider cost accounting separated from page and browser-action units.
- Typed outcomes for target denial, provider exhaustion, region unavailable and challenge detected.
- Domain abuse controls, audit trails, emergency deny lists and an operator kill switch.
- Explicit configuration before JavaScript
evaluatecan reach the public API.
These controls are prerequisites, not optional hardening after launch.