Manage and automate your spiders at scale
Think of it as a Heroku for web data extraction. From the creators of the Scrapy framework.
Host and monitor your Scrapy spiders in the cloud
AI extraction for your spiders
Add Zyte API's AI extraction to your Scrapy Cloud spiders for structured product data without parser maintenance.
Focus on the output
Run, monitor, and control your crawlers with Scrapy Cloud's easy-to-use web interface.
On-demand scaling
Add capacity to your scraping operation with a few clicks.
Easy integration
Scrapy Cloud runs your spiders; Zyte API keeps them unblocked - integrate it with one middleware setting. Most production teams use both, and if your self-hosted runtime already works, Zyte API composes with it either way.
Full suite QA tools
Built-in spider monitoring, logging and data QA tools. Along with easy integration of Spidermon, our open source spider monitoring framework.
Zero vendor lock-in
Develop your code using Scrapy, the most-used open-source web scraping framework, and retain the freedom to migrate it to any hosting solution.
Trusted by data-fueled organizations
Designed for web scraping at scale
Everything you need straight out of the box
Start scraping the web in minutes. Deploy code to Scrapy Cloud via your command line or directly with GitHub
Real-time dashboard
Intelligent scheduling
Built-in monitoring
Customizable containers
Zyte API integration
Zyte API browser rendering support
From self-hosted
Already running Scrapy? Deploy in three commands
Your existing spiders, hosted. No rewrites, no infrastructure - deploy the project you already have and schedule it from the dashboard.
Migrating from self-hosted Scrapy? Start here.
1$ pip install shub
2$ shub login
3$ shub deploy
4# your spider is live - schedule it from the dashboardElastic pricing
Only pay for as much capacity as you need.
Starter
Ideal for small projects or if you simply want to give Scrapy Cloud a try.
Free forever
- Unlimited team members
- Unlimited projects
- Unlimited requests
- 1 hour crawl time
- 1 concurrent crawl
- 7 day data retention
Most popular
Professional
Ideal for developers and companies who want a hassle free way to scrape the web at scale.
from
$9
unit per month
- Unlimited team members
- Unlimited projects
- Unlimited requests
- Unlimited crawl time
- Unlimited concurrent crawl
- Intelligent crawl scheduling
- 120 day data retention
- Personalised support
1 Scrapy Unit = 1 GB of RAM and 1 concurrent crawl
Provides a simple way to run your crawls and browse results
Scrapy is really pleasant to work with. It hides most of the complexity of web crawling, letting you focus on the primary work of data extraction.
Zyte provides a simple way to run your crawls and browse results, which is especially useful for larger projects with multiple developers.
Jacob PerkinsStreamHacker.com