Dataset API — Export Run Results

GET /v1/runs/{runId}/dataset/items is Scrapy.io’s dataset API: download structured rows from a completed (or in-progress) scraper run.

JSON (default)

curl -s "https://api.scrapy.infralyon.com/v1/runs/$RUN_ID/dataset/items?offset=0&limit=100" \
  -H "Authorization: Bearer $SCRAPY_API_KEY" \
  -o rows.json

Envelope: { "data": { "items", "total", "offset", "limit" } }.

CSV

curl -s "https://api.scrapy.infralyon.com/v1/runs/$RUN_ID/dataset/items?format=csv" \
  -H "Authorization: Bearer $SCRAPY_API_KEY" \
  -o results.csv

Useful query options (when supported by the export builder):

Param	Purpose
format	json (default), csv, jsonl, xlsx, xml, html, rss, …
fields / columns	Include only listed fields
omit	Drop listed fields
clean	Drop keys starting with # (Apify-style clean items)
desc	Reverse order
offset / limit	Window rows
attachment	Force download disposition

Example — selected fields as CSV:

curl -s "https://api.scrapy.infralyon.com/v1/runs/$RUN_ID/dataset/items?format=csv&fields=username,followers" \
  -H "Authorization: Bearer $SCRAPY_API_KEY" \
  -o profiles.csv

JSON Lines

curl -s "https://api.scrapy.infralyon.com/v1/runs/$RUN_ID/dataset/items?format=jsonl" \
  -H "Authorization: Bearer $SCRAPY_API_KEY" \
  -o results.jsonl

Large exports

• Prefer pagination (offset / limit) for JSON APIs consumed by code.
• For bulk download into files, format=csv or jsonl is usually easier than paging JSON envelopes.
• Avoid requesting unbounded pages — max limit is 100 for Platform list-style windows.

Related

• Monitor and dataset
• Pagination
• Runs API