PINGDOM_CHECK

Field notes from the world of data extraction.

Articles, interviews and analysis on how data is gathered, used and fought over — written by the people closest to it.

XPath Tips From The Web Scraping Trenches
Scraping practice

XPath Tips From The Web Scraping Trenches

XPath is helpful for web scraping, allowing to write specifications more flexibly than CSS selectors. This tutorial is packed with XPath tips and examples.

Valdir Stumm Junior3 min read
Introducing Data Reviews: Unlocking Insights with Zyte
Product Update

Introducing Data Reviews: Unlocking Insights with Zyte

Introducing Data Reviews - Empower your decision-making with data reviews. Gain insights into data quality and credibility for your web scraping projects.

Valdir Stumm Junior1 min read
Extract Schema.Org Microdata with Scrapy Selectors
How To

Extract Schema.Org Microdata with Scrapy Selectors

Web pages are full of data. Microdata markup helps machines understand pages. Schema.org supports a set of schemas for structured data markup on web pages.

Valdir Stumm Junior5 min read
Autoscraping Casts A Wider Net
Open-source

Portia: The Open-Source Visual Web Scraper

As you can see, Portia allows you to visually configure what’s crawled and extracted in a very natural way. It provides immediate feedback, making the process

Shane Evans1 min read
Aduana: Link Analysis to Crawl the Web at Scale
How To

Optimizing Memory Usage Of Scikit-Learn Models Using Succinct Tries

We use the scikit-learn library for various machine-learning tasks at Zyte. For example, for text classification we'd typically build a statistical

Mikhail Korobov7 min read
Open source at Zyte
Open-source

Open source at Zyte

Open Source at Zyte, Now Zyte - Embrace the open-source movement at Zyte. Learn how we contribute to the community and promote transparency.

Pablo Hoffman2 min read
Chats with Rinar Solutions: Insights into Remote Working
Announcement

Open Source | Zyte | Looking Back At 2013

Looking Back at 2013 - Join us as we reflect on the highlights and achievements of Zyte in 2013. See how far we've come in the web scraping industry.

Shane Evans2 min read
Marcos Campal Is A ScrapingHubber!
Developer interest

Marcos Campal Is A ScrapingHubber!

Marcos Campal is a Zyteber - Get to know one of our talented team members, Marcos Campal. Learn about his contributions to the world of web scraping.

Pablo Hoffman1 min read
Introducing Dash
Product Update

Introducing Dash

Introducing Dash - Discover Dash, a new tool designed to simplify web scraping. Learn how to leverage its capabilities for better data extraction.

Shane Evans4 min read
A Practical Guide to Web Data QA (Part V): Navigating Broad Crawls
Leadership

Why MongoDB Is A Bad Choice For Storing Scraped Data

MongoDB was used early on at Zyte to store scraped data because it's convenient. Scraped data is represented as (possibly nested) records which can be

Shane Evans4 min read
Proxy management: In-house or off-the-shelf proxy solutions?
Product Update

Introducing Smart Proxy Manager

Introducing Zyte Smart Proxy Manager - Enhance your web scraping with Smart Proxy Manager. Explore its powerful features and benefits as a smart proxy manager.

Pablo Hoffman1 min read
4 simple Steps for effective Automated Data QA Process
How To

Git Workflow For Scrapy Projects

Git Workflow for Scrapy Projects - Streamline your Scrapy projects with an efficient Git workflow. Improve collaboration and project management.

Pablo Hoffman2 min read