Python
Articles from the Zyte blog in Python.
Self-healing spiders: letting an agent fix your Scrapy crawl, with a lid on it
Two demos from our Extract Summit Austin workshop: an agent repairs a Scrapy spider after a site redesign, then heals a live crawl mid-run through Spidermon and the Scrapy MCP, with the measured costs and the mistakes that shaped the design.
![JSON Parsing with Python [Practical Guide]](/_next/image/?url=https%3A%2F%2Fextract.zyte.com%2Fapi%2Fmedia%2Ffile%2Fjson-parsing-with-python-hero-1-1200x630-qxGk7q0hXsCKlTSSTGo73YeO5qEHaa.png&w=3840&q=75)
JSON Parsing with Python [Practical Guide]
Parse JSON in Python with json.loads and json.load, read what each error means, and get JSON out of APIs and web pages with JMESPath and ChompJS.
Inside Scrapy's RemoteControl extension
Scrapy 2.19's RemoteControl extension runs a small authenticated HTTP server inside every crawl. How it starts, how it's secured, what the code you send can reach, and what you can do with it on a live crawl, plus the traps and what changes under scrapy-zyte-api.
One Scrapy Spider, Three Browser Setups: Playwright, Patchright, and Zyte
One Scrapy spider, three browser setups: stock Playwright, Patchright via the new PLAYWRIGHT_BROWSER_PROVIDER hook, and a remote browser on Zyte over CDP. Same spider and selectors throughout, only the browser changes.
Stopping a crawl when the data stops looking real with Jev
A Scrapy pipeline that asks a fast, calibrated AI model whether each scraped field still looks real, and stops the crawl when too many don't. What it caught, what it misses, and whether building it was worth it.
How to ensure data quality in your Scrapy web scraping projects using Spidermon and Claude Code
Spidermon is an open-source monitoring framework for Scrapy. You attach it to your spider, define what "success" looks like, and it automatically checks your crawl results after the spider closes, flagging anything that doesn't meet your standards.
The Scrapy whisperer: Adrian Chaves on Web Scraping Copilot
An interview with Scrapy maintainer Adrian Chaves on Zyte’s Web Scraping Copilot, AI-generated parsing code, and building reliable scraping workflows.
Stop using Python requests for web scraping: Use these modern modules instead
While the 'Requests' library remains the default choice for many Python developers due to its reliability and extensive documentation, the Python HTTP landscape has evolved considerably. Modern alternatives now offer significant advantages, including built-in asynchronous support, HTTP/2 compatibility, enhanced performance, and up-to-date TLS handling.
Building a production-style web scraper with Scrapy, Docker, and PostgreSQL
Demo project scrape2postgresql shows how to scrape structured data with Scrapy, store it in PostgreSQL, and run both the spider and database in separate containers using Docker Compose.
Scraping Swiss Army Knife: My personal fix for web setup fatigue using Docker, Scrapy and Zyte
Tired of repeating web scraping setup? Learn how a multi-arch Docker container with Scrapy, Zyte, Requests, and Pandas speeds up exploration and debugging.
How I trade gold using e-ink, live data and an old Raspberry Pi
Track real-world gold and silver retail prices automatically using Zyte API, Python, and a Raspberry Pi with an e-ink display. Learn how to scrape rendered HTML, parse prices, and build an always-on trading dashboard.
Scrapy in 2026: New release brings modern async crawling standards
Scrapy 2.14.0 modernizes the framework with native async/await, smarter scheduling, and cleaner spider configuration. Here’s what the new release means for production crawlers in 2026.