Explore resources by topic or category

If you’ve ever worked on a web scraping project, you’ve most likely heard of a proxy server. But what exactly does a proxy server mean and how does it affect your web scraping project?

If you’re involved in any kind of web data extraction project, you’ve probably heard about headless browser scraping.

In this tutorial, you will learn how to use smart proxy manager to scale up your already existing Scrapy project in order to make more requests and extract more web data.

In this article, we give you some insight on how you can scale up your web data extraction project.

Web scraping is when you extract data from the web and put it in a structured format.

Adding proxies is one of the first actions you will need to take. You will need to manage a healthy proxy pool to avoid bans.

Zyte Smart Proxy Manager is a proxy service, specifically designed for web scraping. In this article, you are going to learn how to use Zyte Smart Proxy Manager inside your Scrapy spider.

Configuring and using proxies is not easy, especially when sending HTTP requests.