Product overview
Scrapy.io is a web scraping API: you call HTTP endpoints with an API key, run marketplace scrapers, and download structured datasets.
This page explains what the product covers. The task path starts at Getting started.
Start here
- Getting started — the short curl walkthrough
- Run a scraper — inputs, sync, and async
- Python or JavaScript if you are not calling HTTP directly
What you can do with the Scrapy.io scraper API
| Capability | Where |
|---|---|
| Discover scrapers / tools | GET https://api.scrapy.infralyon.com/v1/tools |
| Run one result (sync) | POST https://{publisher}.p.scrapy.infralyon.com/{tool}/v1/api |
| Run a batch (async) | POST https://{publisher}.p.scrapy.infralyon.com/{tool}/v1/scraper |
| Poll job status | GET https://api.scrapy.infralyon.com/v1/runs/{runId} |
| Dataset API — export rows | GET https://api.scrapy.infralyon.com/v1/runs/{runId}/dataset/items |
| Schedule recurring scrapes | POST https://api.scrapy.infralyon.com/v1/schedules |
Why teams choose a scraping API
Compared with building your own crawlers, a hosted data extraction API gives you:
- Ready-made tools for Instagram, LinkedIn, Google Maps, TikTok, and more
- Pay-per-result billing (you are charged for successful rows)
- One API key for discovery, runs, datasets, and schedules
- JSON / CSV / JSONL exports for warehouses and AI agents
If you are evaluating an Apify alternative, Scrapy.io keeps a similar developer workflow (run → poll → dataset) with Scrapy.io’s own publisher URLs and Platform API.
Architecture (two hosts)
Web scraping API (docs + Platform) Execution (publisher)
api.scrapy.infralyon.com/v1 {publisher}.p.scrapy.infralyon.com
tools / runs / datasets / schedules /v1/api | /v1/scraper
The Platform API does not execute scrapes; publisher hosts do. That split keeps the platform API stable while tools evolve.
Platform scraper APIs
Search the catalog with GET /v1/tools, then call {publisher}.p.scrapy.infralyon.com/{slug}. Find a scraper shows the URL shape and one sample call.
Discover or filter the same catalog with GET /v1/tools?q=... or the Store.
Field-level contracts are in the API reference. Common questions are in the FAQ.