Docs examples target https://api.scrapy.infralyon.com (not production https://api.scrapy.io).
Skip to main content

Build your first tool

Publish a private Node tool from the Scrapy My tools → Develop New flow. Scrapy builds your source into a Cloudflare Container image; your code talks to the runtime API with a run-scoped token.

What to upload​

RequiredOptional
package.json with scripts.start and @scrapyio/sdk ^0.2.0input_schema.json
src/main.js (entry)README.md
package-lock.json (for npm ci)—

Do not include a Dockerfile. Scrapy generates one from the platform template catalog (node-basic for plain JS).

Minimal entry​

import { Tool, log } from '@scrapyio/sdk';

await Tool.init();
const input = (await Tool.getInput()) ?? {};
const message = typeof input.message === 'string' ? input.message : 'hello-from-starter';
await Tool.pushData([{ message }]);
await Tool.exit({ statusMessage: 'ok' });

Input → output example​

{ "message": "hello-from-demo" }

Run results include an item with that same message string.

Develop New (template)​

  1. Open My tools → Develop New.
  2. Choose Web scraper → Container → a Node template → preview → Use this template.
  3. Click Save. Scrapy creates a private tool you own and seeds the Node starter.
  4. On the tool page, confirm Source (or edit in the Web IDE and Save source).
  5. Open Last build → Build. Wait until status is Ready.
  6. Open Input / Last run → Test run with JSON input. Check results and logs.

Import from Git​

  1. On Develop New, click Import from Git.
  2. Scrapy creates an empty private tool and opens the GitHub import panel.
  3. Connect the Scrapy GitHub App if needed, pick repo / branch / subfolder, import.
  4. Build → Ready → Test run (same as above).

Download starter​

On the Source panel, use Download starter template (scrapy-node-starter.zip), or GET /api/publisher/tools/starter.zip while signed in.

Limits (v1)​

  • Node only (no Python publisher builds yet).
  • Environment variables / secrets for tools are coming soon (K25). Containers never receive DATABASE_URL or account secrets.