Docs examples target https://api.scrapy.infralyon.com (not production https://api.scrapy.io).
Skip to main content

JavaScript SDK

Official TypeScript / JavaScript client for the Scrapy.io Platform API and publisher execution APIs.

Published version: 0.1.1
Package: @scrapyio/sdk

Server-side only. Keep API keys off browsers and client devices.

Install​

npm install @scrapyio/sdk

Initialize​

import { ScrapyIO } from '@scrapyio/sdk';

const client = new ScrapyIO({
  apiKey: process.env['x-api-key']!,
});

The SDK always sends x-api-key: <apiKey>.

Use x-api-key for SDKs and MCP. Raw curl / HTTP docs on this site use SCRAPYIO_API_KEY as a shell variable for the same key value — see Authentication.

Optional constructor fields: baseUrl (default https://api.scrapy.infralyon.com/v1), fetch, maxRetries (default 2), retryBackoffMs (default 250).

Discover tools​

const tools = await client.tools.list({ q: 'instagram', limit: 20 });
// { items, total, offset, limit }

const tool = await client.tools.get('datadoping', 'instagram-profile-scraper');
// or:
const same = await client.tool('datadoping/instagram-profile-scraper').get();

Run a scraper​

Sync — .call()​

const result = await client.tool('datadoping/instagram-profile-scraper').call({
  username: 'nasa',
});

Async — .start() + wait + dataset​

const run = await client.tool('datadoping/instagram-profile-scraper').start({
  usernames: ['nasa'],
});

const finished = await client.run(run.id).wait({
  pollIntervalMs: 3000,
  timeoutMs: 15 * 60_000,
});

const page = await client.run(run.id).listItems({ offset: 0, limit: 100 });
// { items, total, offset, limit }

Also available: client.runs.list(...), client.runs.get(runId), client.run(runId).get().

Schedules​

const schedule = await client.schedules.create(
  {
    publisher: 'datadoping',
    slug: 'instagram-profile-scraper',
    runName: 'nightly-ig',
    timezone: 'UTC',
    frequency: 'one-time',
    date: '2026-09-20',
    time: '23:50',
    inputs: ['nasa'],
  },
  { idempotencyKey: 'nightly-ig-v1' }
);

await client.schedules.update(schedule.id, { isActive: false });
await client.schedules.delete(schedule.id);

Also: client.schedules.list(...), client.schedules.get(id).

Errors​

import { ScrapyIO, ScrapyIOError } from '@scrapyio/sdk';

try {
  await client.tools.get('nope', 'missing');
} catch (err) {
  if (err instanceof ScrapyIOError) {
    // err.type, err.status, err.message, err.docUrl
  }
}

Retries​

GET/HEAD Platform requests may retry on transient 408 / 429 / 5xx (default up to 2 retries with exponential backoff). Execution POSTs (.call() / .start()) and schedule creates are not auto-retried — use idempotencyKey for schedule creates.

Limitations (v0.1)​

  • listItems() is JSON-only (omit format or pass "json"). For CSV/JSONL, use the Platform HTTP dataset export endpoints.
  • Designed for Node / server runtimes (Node 18+ recommended).

Mental model​

client
├── tools.list / tools.get / tools.getReadme / tools.listReviews
├── account.get / account.usage
├── tool("publisher/slug").get / .call / .start
├── runs.list / runs.get / runs.abort / runs.resume
├── run(id).get / .wait / .listItems / .abort / .resume
└── schedules.list / get / create / update / delete