Python
Articles from the Zyte blog in Python.
How to build your first Scrapy extension
Why my Scrapy project plays a triumphant fanfare when a crawl finishes clean and a sad trombone when it doesn't, and how I finally learned how to build Scrapy extensions (it's easy)
Automate deployment of your web scraper on VPS with Ubuntu 24.04 cloud-init
Your VPS is ready, but now you need to work through the same sequence you have run a dozen times before: apt update, apt install python3-pip, pip install scrapy, playwright install chromium, the Chromium dependency list that never installs cleanly on the first try, Redis, possibly Postgres, whatever else this particular project needs.
Flatcar Linux for web scrapers: deploy immutable containers with just one config file
the next time you spin up a VPS to give it a persistent home, you spend the better part of an afternoon rebuilding from memory: installing Scrapy, wiring up Redis, configuring the systemd units, getting Playwright's Chromium dependencies in the right state. Here's a tool to help
The Scrapy whisperer: Adrian Chaves on Web Scraping Copilot
An interview with one of the popular framework’s co-maintainers on Zyte’s new, AI-powered Scrapy sidekick.
Introducing Web Scraping Copilot 1.0: AI-accelerated web scraping inside VS Code
Discover Web Scraping Copilot 1.0, Zyte’s VS Code extension that uses AI to generate, test, and deploy production-ready Scrapy spiders faster while maintaining full developer control.
Scraping Swiss Army Knife: My personal fix for web setup fatigue using Docker, Scrapy and Zyte
Tired of repeating web scraping setup? Learn how a multi-arch Docker container with Scrapy, Zyte, Requests, and Pandas speeds up exploration and debugging.
How I trade gold using e-ink, live data and an old Raspberry Pi
Track real-world gold and silver retail prices automatically using Zyte API, Python, and a Raspberry Pi with an e-ink display. Learn how to scrape rendered HTML, parse prices, and build an always-on trading dashboard.
Scrapy in 2026: New release brings modern async crawling standards
Scrapy 2.14.0 is released with a major under-the-hood modernization. Say goodbye to Twisted Deferreds.
.png&w=3840&q=75)
Introducing Web Scraping Copilot - A rocket boost for data extractors
Meet Web Scraping Copilot, a free VS Code extension that uses AI to accelerate Scrapy projects. Generate code, manage spiders, and deploy to Scrapy Cloud faster than ever, keeping developers in control.
The future of Scrapy: Smarter, faster and ready for AI-powered scraping
What does the future hold for the tool some describe as “the gift that revolutionised web scraping”?
Ten years since Scrapy 1.0: The stats and stories behind your favorite framework
See what 10 years of Scrapy 1.0 has produced — in milestones and metrics - as it became the most-used open source web scraping framework in the world.
The rise of Scrapy: How an open-source scraping framework conquered the web
The story of Scrapy reflects the broader evolution of the web itself and the ongoing quest to harness its ever-expanding ocean of information.
_HFpro5d6k3.png&w=256&q=75)
_E4PyVpfAxa.png&w=256&q=75)
-(1).png&w=1920&q=75)
-(1)_VZGHqxCgXV.png&w=1920&q=75)
