PINGDOM_CHECK

Field notes from the world of data extraction.

Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.

Web Scraping Price Monitoring
Use case

Web Data Analysis: Exposing NFL Player Salaries With Python

What are the salaries of the NFL's biggest players? Find out how with Python web scraping and web data analytics.

Attila Toth10 min read
From the creators of Scrapy: Automatic data extraction API

From the creators of Scrapy: Automatic data extraction API

Artificial Intelligence Data Extraction API - Explore the power of an AI-driven data extraction API, making data retrieval smarter and more efficient.

4 min read
Zyte A.I. Automatic Extraction API | e-commerce & articles
AI-assisted data extraction

Zyte A.I. Automatic Extraction API | e-commerce & articles

AI-powered Automatic Extraction API for e-commerce & article extraction. capable of automatically identifying common items on web pages to extract.

Ian Kerins3 min read
Building Spiders Made Easy | GUI For Scrapy Shell
How To

How to define the scope of your web scraping project

Interested in starting a web scraping project? In this article, we walk you step-by-step through how you should define the scope of your web scraping project.

Colm Kenny8 min read
4 Sectors That Benefited Most From Business Intelligence Software
Scraping strategy

The best way to architect web scraping solutions | Zyte

Architect web scraping solutions with our step-by-step guide and optimize your data projects. Learn to use the right technologies that create the best results.

Colm Kenny6 min read
The rise of web data in hedge fund decision making
Web data collection ethics

Navigating compliance when extracting alternative data for finance

Maintain regulatory compliance when using alternative web-scraped financial data for business requirements.

Sanaea Daruwalla10 min read
How to use XPath to extract web data
Data quality

Spidermon: Zyte's secret to data quality

Unveil Spidermon's role in our data quality assurance. Elevate confidence in the reliability of your web-scraped data.

Ian Kerins5 min read
How to use XPath to extract web data
Open-source

Meet Spidermon: Our battle tested spider monitoring library

Learn how to easily monitor scrapy spiders & validate data with Spidermon! Developed by Zyte Spidermon is now available as an open-source library.

Renne Rocha6 min read
Proxy management: In-house or off-the-shelf proxy solutions?
Proxies

Proxy management: In-house or off-the-shelf proxy solutions?

This article helps you decide whether you should handle your proxy management in house or by using third party proxy solutions like scraping APIs.

Ian Kerins7 min read
Backconnect proxies explained: How to use them in a scraping project?
Proxies

A sneak peek inside Zyte Smart Proxy Manager

Take a look behind the scenes inside Smart Proxy Manager , the world's smartest web scraping proxy network.

Pablo Hoffman6 min read
Backconnect proxies explained: How to use them in a scraping project?
Product Update

The Zyte Smart Proxy Manager Story: Enhancing Scraping Efficiency

Discover Smart Proxy Manager (Crawlera), the world's smartest proxy network tailored for web scraping, eliminating proxy management hassles.

Pablo Hoffman6 min read
The rise of web data in hedge fund decision making
Web data application

The rise of web data in hedge fund decision making

Learn about the rise of web data in hedge fund decision making and the importance of data quality in this blog post.

Ian Kerins8 min read