PINGDOM_CHECK

Field notes from the world of data extraction.

Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.

Unblockers vs Zyte API: What’s the Real Cost of Bans?
Anti-ban

Unblockers vs Zyte API: What’s the Real Cost of Bans?

Not all unblockers are equal. Learn how Zyte API automates unblocking and why predictable success rates save more than cheap bandwidth.

Mitch Holt10 min read
AI and the web: What 2025 changed and what comes next
Web data collection

Partial autonomy, full control: Why we built Web Scraping Copilot

Web Scraping Copilot brings AI-powered, code-first web scraping into VS Code, delivering partial autonomy, deterministic quality, and full control for production web data.

Iain Lennon10 min read
Why the Best Engineers Are Actually Lazy
Anti-ban

Why your spiders keep getting banned, and how to fix it

Learn why scrapers get blocked and how modern unblocking APIs like Zyte API automate ban handling for faster, more reliable data extraction.

Mitch Holt10 min read
Introducing Web Scraping Copilot - A rocket boost for data extractors
Web scraping APIs

New in Zyte: Web Scraping Copilot, LLM-friendly text, MCP scraping

Discover Zyte’s latest: Web Scraping Copilot, LLM-ready page content, new Zyte API features, enterprise tools, webinars, and Extract Summit talks on-demand.

Daniel Cave5 min read
AI and the web: What 2025 changed and what comes next
Web data collection

Five key takeaways from Extract Summit 2025

From AI-accelerated scraping to “dead internet” risks and rising access wars, these five takeaways from Extract Summit 2025 show where web data is heading next.

Robert Andrews5 min read
Introducing Web Scraping Copilot - A rocket boost for data extractors
Web data collection

Introducing Web Scraping Copilot - A rocket boost for data extractors

Meet Web Scraping Copilot: your AI-powered coding partner for Scrapy in VS Code.

Valter Sciarrillo10 min read
AI and the web: What 2025 changed and what comes next
Web data collection ethics

Web scraping as social practice: Ethics and efficiency in a data-hungry world

Web scraping is more than a technical act, it’s a social practice. Explore ethical scraping principles, responsible data collection, and how to balance efficiency with respect for data and people.

Rodrigo Silva Ferreira10 min read
Buy or Build? The Four Roads to Acquiring Web Data
AI-assisted data extraction

Extract clean content automatically with Zyte API’s new pageContent data type

Discover how Zyte API’s new PageContent data type makes content extraction effortless — delivering clean, structured data from any web page automatically.

Daniel Cave10 min read
Balancing innovation and regulation in data scraping
Web data collection legality

Balancing innovation and regulation in data scraping

Weighing your options from full control to full service.

Sanaea Daruwalla10 min read
The new economics of web data: Smaller scraping just got cheaper
Scraping strategy

The new economics of web data: Smaller scraping just got cheaper

Discover how AI and new scraping tech are lowering costs, removing the “scale tax,” and making web data access affordable for smaller teams.

Theresia Tanzil2 min read
Why your agent deserves a wallet
AI

Why your agent deserves a wallet

Acting autonomously on tasks like data gathering, financially-empowered agents could change the economics of the web.

Jan Seidler2 min read
Memo to CTOs: Don’t build the product
Developer interest

Memo to CTOs: Don’t build the product

Zyte’s CTO says great tech leaders don’t work on software, they work systems that make software.

Jan Seidler2 min read