Field notes from the world of data extraction.
Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.
Should you BUY your WEB DATA, or Write Code?
Stop Scraping RAW HTML into your LLM. Try our new feature
How I go from Zero to Data in 10 minutes with Scrapy
Gemini 3.0 Pro Code Gen With Web Scraping Copilot
Build Your Own Price Drop Notifier (Zyte Auto Extract + IFTTT Mobile Alerts)
Hybrid Scraping: The Architecture for the Modern Web
Modern Web Scraping starts with THIS.
3 Rules of Modern Web Scraping
The Modern Scrapy Developer's Guide (Part 3): Auto-Generating Page Objects with the Web Scraping Copilot
In this guide, we'll show you how to use Web Scraping Copilot (our VS Code extension) to automatically write 100% of your Items, Page Objects, and even your unit tests.
The Modern Scrapy Developer's Guide (Part 2): Page Objects with scrapy-poet
In this guide, we'll fix this by refactoring our spider to a professional, modern standard using Scrapy Items and Page Objects (via crapy-poet). We will completely separate our crawling logic from our parsing logic.
The Modern Scrapy Developer's Guide (Part 1): Building Your First Spider
In this definitive guide, we will walk you through, step-by-step, how to build a real, multi-page crawling spider. You will go from an empty folder to a clean JSON file of structured data in about 15 minutes
Zyte’s 2025 review: Year of locks and unlocks
2025 reshaped web scraping. From AI-assisted extraction and escalating bot defenses to clearer legal frameworks and cheaper APIs, Zyte reviews the forces redefining access to web data—and what comes next.