Explore resources by topic or category

Whether you're scraping a simple webpage or navigating a complex multi-step process, leveraging sessions is key to ensuring success.

Understanding how to bypass IP bans is essential to anyone who wants to collect web data at any scale.

Web scraping tools save hours of work by automating data extraction, testing web applications, and performing repetitive tasks.

In this article, I’llexplain the problem of anti-bot technology for web scraping developers through the lens of the anti-bot distribution curve (a view of the top 250,000 websites and the relative complexity of their anti-bot tech) and the landscape of anti-bot tech across the web.

Web scraping developers often find themselves in a struggle to manage bans and blocks. Every time they resolve a ban, it's only a matter of time before their scrapers encounter the same issue again.

Web scraping challenges, ranging from IP bans and data accuracy to legal compliance issues, can trip up businesses trying to use web data to fuel machine learning and to make better decisions.

We have launched a new Zyte SmartProxy Playwright and we’re sure you’re going to love it!

If you are new to proxies, we recommend skimming through this blog to understand what a proxy is and why you need proxies for web scraping.

If you’re involved in any kind of web data extraction project, you’ve probably heard about headless browser scraping.

In this tutorial, you will learn how to use smart proxy manager to scale up your already existing Scrapy project in order to make more requests and extract more web data.