Explore resources by topic or category

Most AI skills are never tested — and it shows. Here's how Zyte evaluates scraping skills like real software, catching failures demos miss.

In our interview, a QA expert warns - before you delegate web scraping quality assurance to AI, make sure you can describe what ‘good’ looks like for yourself.

While the 'Requests' library remains the default choice for many Python developers due to its reliability and extensive documentation, the Python HTTP landscape has evolved considerably. Modern alternatives now offer significant advantages, including built-in asynchronous support, HTTP/2 compatibility, enhanced performance, and up-to-date TLS handling.

What does the future hold for the tool some describe as “the gift that revolutionised web scraping”?

See what 10 years of Scrapy 1.0 has built — in milestones and metrics.

The story of Scrapy reflects the broader evolution of the web itself and the ongoing quest to harness its ever-expanding ocean of information.

Learn how successful open-source projects balance community value with sustainable growth. Industry leaders share insights on monetization, maintenance, and building thriving communities.

Discover how Zyte’s open-source libraries like ClearHTML, Extruct, Chomp.js, and more simplify web data extraction and processing.

Here are four essential Scrapy plugins we use to build efficient web crawlers for our customers.

Learn about the scrapers system: Explorer’s Compass to analyze websites.

Everything you need to know about python web scraper tools and libraries including Requests, BeautifulSoup, Selenium and Scrapy.

Get the best value for your web crawling project by using Scrapy. An awesome framework you should learn and incorporate for easy and accurate web crawling.