Docs examples target https://api.scrapy.infralyon.com (not production https://api.scrapy.io).
Skip to main content

FAQ

Answers to common questions about the Scrapy.io web scraping API, Platform API, and scraper marketplace. Optimized for search queries such as “how to use scraper API”, “web scraper API Python”, and “Apify alternative”.

What is Scrapy.io?​

Scrapy.io is a scraper marketplace with a public Platform API (api.scrapy.infralyon.com) and publisher execution APIs ({publisher}.p.scrapy.infralyon.com). You discover tools, run scrapers with an API key, and export datasets.

What is a web scraping API?​

A web scraping API (or web scraper API) lets you request website data over HTTP instead of running your own crawlers. Scrapy.io exposes sync and async endpoints plus a dataset API to download results as JSON or CSV.

How do I use the scraper API?​

  1. Create an API key in the console.
  2. Call GET https://api.scrapy.infralyon.com/v1/tools to find a tool.
  3. Execute on https://{publisher}.p.scrapy.infralyon.com/{slug}/v1/api or /v1/scraper.
  4. Poll GET /v1/runs/{runId} and fetch GET /v1/runs/{runId}/dataset/items.

Full walkthrough: Getting started and Web scraping API.

Can I use the web scraper API with Python?​

Yes. Use any HTTP client (requests, httpx) with:

Authorization: Bearer scrapy_api_...

Example:

import requests

r = requests.get(
    "https://api.scrapy.infralyon.com/v1/tools",
    headers={"Authorization": f"Bearer {API_KEY}"},
    params={"q": "maps", "limit": 10},
    timeout=60,
)
print(r.json()["data"]["items"])

See Run a tool for sync and async examples.

Where is the scraper API documentation / OpenAPI?​

  • Human docs: this site (https://docs.scrapy.io)
  • OpenAPI: https://docs.scrapy.io/openapi/platform-v1.yaml
  • Platform reference: Platform overview

What is the Platform API vs the publisher API?​

APIHostPurpose
Platform APIapi.scrapy.infralyon.com/v1Tools catalog, runs, datasets, schedules
Publisher execution{publisher}.p.scrapy.infralyon.comStart sync /v1/api or async /v1/scraper jobs

How does the dataset API work?​

GET /v1/runs/{runId}/dataset/items returns paginated JSON by default. Add format=csv or format=jsonl to export files. Details: Export dataset and Runs.

How does scraper API pricing work?​

Scrapy.io uses pay-per-result pricing. Each tool has a pricePerResult. Successful rows are metered against your credits. Create-schedule and start-run paths check credits before work begins. See the pricing page on the main site for plan details (scraper api pricing intent).

Is Scrapy.io an Apify alternative?​

Yes — many teams compare Scrapy.io as an Apify alternative. Both offer run → poll → dataset developer workflows. Scrapy.io uses its own hosts (api.scrapy.infralyon.com and {publisher}.p.scrapy.infralyon.com), pay-per-result marketplace tools, and does not require you to deploy Actors on Apify Cloud.

Scrapy.io documentation is original and does not copy Apify’s API surface.

How do I schedule scrapes?​

Use POST /v1/schedules with publisher, slug, calendar fields, and inputs. Guide: Schedules.

How do I authenticate?​

Prefer:

Authorization: Bearer scrapy_api_...

X-API-Key is also accepted. Never put keys in query strings. See Authentication.