Scrapy Cloud

Scalable cloud hosting for your Scrapy spiders

Host and monitor your Scrapy spiders in the cloud

Manage and automate your spiders at scale

Think of it as a Heroku for web data extraction. From the creators of the Scrapy framework.

AI Scraping

Scrapy, Scrapy Cloud, and Zyte API work seamlessly to bring mature enterprise-grade AI needed to extract e-commerce product data reliably, accurately, and at scale.

Learn more

Focus on the output

Run, monitor, and control your crawlers with Scrapy Cloud's easy-to-use web interface.

On-demand scaling

Increase the scale and firepower of your scraping operation with only a few clicks.

Easy integration

Seamlessly integrate Zyte API to your web scraping stack to take the hassle out of scraping the web at scale.

Full suite QA tools

Built-in spider monitoring, logging and data QA tools. Along with easy integration of Spidermon, our open source spider monitoring framework.

Zero vendor lock-in

Develop your code using Scrapy, the most popular open-source web scraping framework, and retain the freedom to migrate it to any hosting solution.

Trusted by data driven organizations

Designed for web scraping at scale

Everything you need straight out of the box

Start scraping the web in minutes. Deploy code to Scrapy Cloud via your command line or directly with GitHub

Real-time dashboard
Intelligent scheduling
Built-in monitoring
Customizable containers
Smart Proxy Manager integration
API headless browser integration

Elastic pricing

Only pay for as much capacity as you need.

Starter

Free forever

Ideal for small projects or if you simply want to give Scrapy Cloud a try.

Try free

Unlimited team members
Unlimited projects
Unlimited requests
1 hour crawl time
1 concurrent crawl
7 day data retention

Professional

from

/ unit per month

Ideal for developers and companies who want a hassle free way to scrape the web at scale.

Try free

Unlimited team members
Unlimited projects
Unlimited requests
Unlimited crawl time
Unlimited concurrent crawl
Intelligent crawl scheduling
120 day data retention
Personalised support

1 Scrapy Unit = 1 GB of RAM and 1 concurrent crawl

Provides a simple way to run your crawls and browse results

Scrapy is really pleasant to work with. It hides most of the complexity of web crawling, letting you focus on the primary work of data extraction.

Zyte provides a simple way to run your crawls and browse results, which is especially useful for larger projects with multiple developers.

Jacob Perkins

StreamHacker.com