Built for real-world scraping

A set of skills that teach coding agents how to build production-ready web scrapers. Point an agent at a URL, describe the data you want, and it explores the site, proposes a schema for your approval, then generates a complete Scrapy project with web-poet page objects — no manual scaffolding required. It's available in three forms so you can use it in whichever coding agent you already work in:

• Claude Code plugin — for Claude Code specifically.

• Universal Skills — the same skills, packaged to the open Agent Skills standard, so they work with any coding agent that supports it (Cursor, Cline, and others).

• GitHub Copilot CLI plugin — the same functionality, packaged for Copilot CLI.

Claude Code has the deepest integration. The Codex plugin is now available, the GitHub Copilot CLI plugin works in your terminal, and Agent Skills bring the same expertise to other AI coding agents. A Cursor plugin is coming soon.

They all run the same underlying skills and produce the same output — a working Scrapy project with page objects and tests. The difference is purely packaging and agent-specific optimizations (e.g. usage of sub-agents):

• Use the Claude Code plugin if you work in Claude Code.

• Use the Copilot CLI plugin if you work in GitHub Copilot CLI.

• Use Universal Skills if you work in any other coding agent that supports the Agent Skills standard (or if you want one install that isn't tied to a specific agent).

You won't get different results by picking one over another — pick based on which agent you already use.

Built for developers and data teams who need reliable, production-ready web data fast, regardless of which coding agent they've standardized on. It's especially useful if you want Scrapy + web-poet scaffolding without writing boilerplate from scratch, a reusable extraction spec before committing to any specific implementation, or a "data-first" workflow where you validate what you're extracting before generating code.

That depends on which package you use:

• Claude Code plugin: Claude Code (CLI or desktop app) and the uv Python package manager (the plugin will prompt you to install it if it's missing).

• Copilot CLI plugin: GitHub Copilot CLI and uv.

• Universal Skills: a coding agent that supports the Agent Skills standard, and uv.

A Zyte API key is recommended for all three — it handles fetching and rendering pages — but isn't strictly required to run any of them.

No, all three can run without one. We recommend using the Zyte API for fetching and extraction, though — it handles JavaScript rendering, anti-bot measures, and reliability, giving you significantly better results on complex sites.

Yes, all three are free to use. As above, we recommend pairing them with the Zyte API for fetching and extraction, which is billed separately based on usage.

The process has two clearly separated stages, regardless of which agent you're using. First, the extraction spec: the agent fetches pages, discovers fields, and iterates with you until you approve the schema and expected values. Second, code generation: the approved spec is used to produce a complete Scrapy project (page objects, test fixtures, spider).

This stage does not crawl the web — it works entirely from the outputs of stage 1. Crawling only happens when you run the generated spider locally, or deploy and run it on Scrapy Cloud.

You get two artifacts: a .scrape/{site-name}/ folder containing the extraction spec (schema, saved pages, and approved expected values), and a fully generated Scrapy project with web-poet page objects, pytest fixtures, and a wired-up spider ready to run — including an option to deploy to Scrapy Cloud. This is identical whether you generated it from Claude Code, Copilot CLI, or another agent via Universal Skills.

Yes. The spec is a technology-neutral artifact that isn't tied to any framework or agent. It can feed a plain-Python implementation, direct Zyte API extraction, or a fully manual process. The generated Scrapy project is one possible consumer — not the only one.

The spec is the durable part of your project. When a site changes, you re-run the spec stage (or update field selectors in the page objects) without discarding the rest of your project structure. The deliberate separation of spec from generated code is what makes the system resilient to change.

Yes. Page fetching and rendering is delegated to the Zyte API, so JavaScript-rendered content is handled transparently — no extra configuration needed, on any of the three packages.