BypassAPI Documentation

An AI web scraper API that bypasses Cloudflare, DataDome and OpenResty, renders JavaScript with a real stealth browser, and returns clean text, Markdown, structured JSON or AI-extracted data.

Introduction

BypassAPI exposes a small, predictable REST API. You send a URL, we open it in a real hardened browser (bypassing bot protection), and return the content in the format you want. Everything is one HTTP call — no browser to run, no proxies to manage, no anti-bot cat-and-mouse.

Base URL for all requests:

https://bypassapi.com

Authentication

Authenticate every request with your API key in the X-API-Key header. Get a key with 25 free credits at bypassapi.com — no credit card required.

X-API-Key: bpa_live_xxxxxxxxxxxxxxxxxxxx

Quickstart

Scrape any page in one call:

curl -X POST https://bypassapi.com/scrape \
  -H "X-API-Key: $BYPASSAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "render": true}'

You get back the title, clean text, raw HTML and LLM-ready Markdown in one response. Add an extract_prompt and the AI returns exactly the fields you asked for.

Credits & pricing

Pay per credit, no monthly subscription. Credits never expire.

Packages start at €17 for 5,000 calls. Failed requests (5xx / bot-block we could not bypass) are never charged — pay only for successful scrapes.

POST /scrape

Fetch and render a URL, return content. This is the main endpoint. Minimal body:

{ "url": "https://target.com/page" }

With rendering, screenshot and AI extraction:

{
  "url": "https://target.com/product/42",
  "render": true,
  "screenshot": true,
  "extract_prompt": "return the product name, price and availability",
  "wait_for_selector": ".price",
  "use_proxy": true
}

POST /extract

Same as /scrape but focused on AI extraction — describe what you want in plain language (extract_prompt is required) or pass an output_schema for strict JSON.

{
  "url": "https://news.ycombinator.com",
  "extract_prompt": "return the top 5 story titles and their point counts as a JSON list"
}

With a strict schema:

{
  "url": "https://target.com/job/123",
  "extract_prompt": "extract the job posting",
  "output_schema": {"title":"string","company":"string","salary":"string","remote":"boolean"}
}

POST /recipe/{name}

Pre-built, battle-tested extraction recipes for specific site families — they handle the site-specific selectors, pagination and auth for you. Available recipes: adult, adulttime, bizreg, propisi.

curl -X POST https://bypassapi.com/recipe/bizreg \
  -H "X-API-Key: $BYPASSAPI_KEY" -H "Content-Type: application/json" \
  -d '{"url": "https://bizreg.example/company/123"}'

POST /crawl

Crawl a whole site from a start URL. We follow same-domain links breadth-first up to your limits and return clean Markdown for each page — ideal for building an LLM knowledge base from an entire docs site or catalog.

{
  "url": "https://target.com",
  "max_pages": 25,
  "max_depth": 2,
  "same_domain": true,
  "extract_prompt": "return the page title and summary"
}

Response: { start_url, count, credits_charged, pages: [{url, title, markdown, status_code, extracted_data?}] }. You are only charged for pages actually crawled — the crawl auto-stops at your credit balance.

POST /batch

Scrape many URLs in a single call (up to 50). Returns the same result object as /scrape for each URL, with optional AI extraction.

{
  "urls": ["https://a.com/1", "https://a.com/2", "https://b.com/x"],
  "extract_prompt": "return the H1 and price"
}

Response: { count, credits_charged, results: [ ScrapeResult, ... ] }.

POST /map

Fast URL discovery for a domain — sitemap.xml + robots.txt + one-hop link graph, without scraping each page. The natural first step before a crawl or an agent run. 1 credit.

{ "url": "https://target.com", "limit": 200, "include": ["/blog/"], "exclude": ["/tag/"] }

Response: { url, count, urls: [...] }. include/exclude are regex filters; same_domain defaults true.

Async jobs & webhooks

For large crawls, pass a webhook URL to /crawl. The call returns immediately with a job_id; we run the crawl in the background and POST the full result to your webhook when done (with retries). Poll GET /jobs/{job_id} any time for status.

POST /crawl { "url": "https://target.com", "max_pages": 200, "webhook": "https://you.com/hook" }
-> { "job_id": "job_ab12...", "status": "queued", "poll": "/jobs/job_ab12..." }

GET /jobs/job_ab12...  -> { "status": "completed", "result": { count, pages: [...] } }
Webhook payload -> { "job_id", "event": "completed", "count", "pages": [...] }

Parameters

All parameters below apply to /scrape and /extract.

Response format

{
  "url": "https://example.com",
  "status_code": 200,
  "title": "Example Domain",
  "raw_text": "Example Domain\nThis domain is for use in...",
  "raw_html": "<!doctype html>...",
  "markdown": "# Example Domain\n\nThis domain is for use in...",
  "extracted_data": { "price": "$42.00" },
  "screenshot_path": "https://bypassapi.com/static/shots/ab12.png",
  "duration_ms": 1840,
  "error": null,
  "metadata": { "captured": [], "proxy_country": "US" }
}

Cloudflare / anti-bot bypass

The engine drives a real Chromium with a natural fingerprint, human-like timing and stealth patches. It passes Cloudflare, DataDome and OpenResty challenge pages where plain HTTP clients get a 403. Combined with residential IPs (use_proxy), it reaches heavily protected targets.

AI extraction

Instead of brittle CSS selectors and XPath, describe the data in plain language. Pass extract_prompt for freeform, or output_schema to force a strict shape. The AI reads the rendered page and returns JSON in extracted_data.

Markdown output

Every response includes a markdown field — the page converted to clean Markdown with scripts, styles, nav and boilerplate removed. Perfect for feeding LLMs, RAG pipelines and AI agents. No extra flag needed.

CAPTCHA solving

Set solve_captcha to hcaptcha, turnstile or recaptcha and the engine auto-detects and solves the challenge before continuing. Solved challenges cost +1 credit.

Proxy & geotargeting

Set use_proxy: true to route through our rotating residential pool, reducing rate-limit bans on large jobs. To target a country, pass a region hint via headers or bring your own proxy with proxy.

Cookies & sessions

Scrape logged-in pages by passing a cookies jar, or persist a browser session with storage_state_path. Use post_load_click to dismiss consent banners or trigger “load more” before extraction.

Screenshots

Set screenshot: true to get a full-page PNG. The URL is returned in screenshot_path.

Network capture

Pass capture_patterns (e.g. ["\\.m3u8", "\\.vtt"]) to capture matching network responses — ideal for media manifests, subtitle tracks and hidden API calls. Captured items land in metadata.captured.

Code examples

curl -X POST https://bypassapi.com/scrape \
  -H "X-API-Key: $BYPASSAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://target.com","render":true,
       "extract_prompt":"return the main heading and all prices"}'

import requests

r = requests.post(
    "https://bypassapi.com/scrape",
    headers={"X-API-Key": "YOUR_KEY"},
    json={"url": "https://target.com", "render": True,
          "extract_prompt": "return the main heading and all prices"},
    timeout=90,
)
data = r.json()
print(data["markdown"])          # LLM-ready markdown
print(data["extracted_data"])    # AI result

const res = await fetch("https://bypassapi.com/scrape", {
  method: "POST",
  headers: { "X-API-Key": process.env.BYPASSAPI_KEY, "Content-Type": "application/json" },
  body: JSON.stringify({ url: "https://target.com", render: true,
    extract_prompt: "return the main heading and all prices" }),
});
const data = await res.json();
console.log(data.markdown, data.extracted_data);

$ch = curl_init("https://bypassapi.com/scrape");
curl_setopt_array($ch, [
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_POST => true,
  CURLOPT_HTTPHEADER => ["X-API-Key: YOUR_KEY", "Content-Type: application/json"],
  CURLOPT_POSTFIELDS => json_encode(["url" => "https://target.com", "render" => true]),
]);
$data = json_decode(curl_exec($ch), true);
echo $data["markdown"];

Errors

Rate limits

Free keys: 5 concurrent requests. Paid keys scale with your package. Every response includes timing in duration_ms; heavy render + AI calls typically complete in 2–6 seconds.

Self-host

Need data to never leave your infrastructure? Run the whole stack on your own server — same REST API, your control, no vendor lock-in. Docker:

docker compose up -d      # FastAPI + Playwright engine on :8077

Contact us for self-host licensing and on-prem support.