PINGDOM_CHECK

#ExtractSummit2026 The world's largest web scraping conference returns. Austin Oct 7–8 · Dublin Nov 10–11

Register now
Data Services
Login
Try Zyte APIContact Sales
  • Unblocking and Extraction

    Zyte API

    The ultimate API for web scraping. Avoid website bans and access a headless browser or AI Parsing

    Ban Handling

    Headless Browser

    AI Extraction

    SERP

    Enterprise

    DocumentationSupport

    Hosting and Deployment

    Scrapy Cloud

    Run, monitor, and control your Scrapy spiders however you want to.

    Coding Agent Add-Ons

    Agentic Web Data

    Plugins that give coding agents the context to build production Scrapy projects. Starts with Claude Code.

  • Data Services
  • Zyte API

    Zyte Data

    Scrapy Cloud

  • Browse

    • BlogArticles, podcasts, videos
    • Case studiesCustomer outcomes
    • White papersIn-depth reports
    • DocumentationGuides & API reference
    • EventsConferences, webinars, recordings

    Subscribe

    • NewsletterSwiftly delivered
    • Join our community2,000+ web scraping engineers
  • Product and E-commerce

    From e-commerce and online marketplaces

    Data for AI

    Collect and structure web data to feed AI

    Job Posting

    From job boards and recruitment websites

    Real Estate

    From Listings portals and specialist websites

    News and Article

    From online publishers and news websites

    Search

    Search engine results page data (SERP)

    Social Media

    From social media platforms online

  • Meet Zyte

    Our story, people and values

    Contact us

    Get in touch

    Support

    Knowledge base and raise support tickets

    Terms and Policies

    Accept our terms and policies

    Open Source

    Our open source projects and contributions

    Web Data Compliance

    Guidelines and resources for compliant web data collection

    Affiliate Program

    Join Zyte’s affiliate program and start earning commissions today

    Join the team building the future of web data
    We're Hiring
    Trust Center
    Security, compliance & certifications
Login
Try Zyte APIContact Sales
G2
Capterra
Proxyway
Search & SERP data

SERP data collection, built for how search works now

When bulk retrieval disappeared and JavaScript rendering became mandatory, the cost of collecting search results at depth jumped for everyone. Zyte's coordinated pagination and residential proxy infrastructure absorbs that complexity. Your unit economics don't collapse every time the rules change.

Talk to our Data Experts
Try Zyte API
The basics

What is SERP data?

A search engine results page is the ranked list of organic results, ads, and rich features (AI Overviews, Knowledge Panels, local packs) that a search engine returns for a query. SERP data is a structured record of that page: which URLs rank where, what snippets appear, and what AI-generated content is shown. It is collected from live search queries, normalized into a consistent schema, and delivered as structured JSON. Because the same keyword returns different results depending on geo, device, language, and time, the value is not just in what ranks. It's in the context that surrounds it.
Who uses it

Use cases across industries

The same data type, put to work differently. Ordered by how directly it applies.

SEO & rank-tracking platforms

Daily keyword rank tracking and visibility monitoring across regions, devices, and SERP feature types.
Millions of keywords tracked daily

Competitive intelligence

Share-of-voice analysis, competitor ranking history, and SERP feature ownership across an entire category.
Full SERP snapshot, not just rank

Enterprise data teams

SERP data as an upstream signal feeding marketing attribution, demand forecasting, and brand monitoring.
Delivered as structured JSON feed

AI search visibility

Monitoring how brands appear in AI Overviews — which sources are cited, how descriptions are framed, and where competitors are favoured.
AI Overview content captured

Content & SEO agencies

Keyword gap analysis, content opportunity research, and before/after ranking validation for client campaigns.
Multi-client, multi-region coverage

Full-page extraction pipelines

SERP as a gateway: find ranked URLs, then follow them with Zyte API's extraction layer — one platform, one pipeline.
No vendor switching
Try Zyte APITalk to our Data Experts
The hard part

Why SERP data is hard to collect at scale

The problem is rarely a single keyword. It is collecting millions of them reliably, at depth, while the rules of collection keep changing underneath you.

Bulk retrieval parameters were removed

A single parameter once returned 100 results in one request. Its removal meant that the same depth now requires up to ten sequential paginated calls — each one a new browser session, a new proxy, a new opportunity to be blocked. For a platform tracking hundreds of thousands of keywords to page 10, that is not a theoretical cost increase. It is a real one, and it landed overnight.

JavaScript rendering is now mandatory

Modern SERPs, especially those serving AI Overviews, don't exist in the raw HTML. They render client-side, which means a plain HTTP request returns a shell, not a page. Reliable collection now requires a real browser with a residential proxy behind it. That is a fundamentally more expensive operation than what most pipelines were built to do.

Anti-bot enforcement scales with your volume

The more you collect, the more you look like an automated client. Search engine defences are dynamic: what works at 10,000 requests a day may fail at 10 million. Teams that built their own proxy rotation find that thresholds shift without notice, turning a working pipeline into a stream of failed requests and CAPTCHAs. The only durable answer is infrastructure that adapts faster than the blocks.

AI Overviews changed what a SERP is

LLM-generated answer blocks now appear above organic results for many queries. Teams that don't capture them are tracking an incomplete picture — and missing the placement their clients increasingly care about most. Zyte returns AI Overview content within the response HTML so your pipeline sees the full page, not just the links beneath it.
The cost of getting it wrong

What poor SERP data quietly costs the business

When bulk retrieval parameters were removed and rendering requirements tightened, teams that absorbed the change quietly are still paying for it. Bad SERP data doesn't announce itself. It shows up later, in a rank report that was already wrong, or a margin that's been silently compressing for months.
Margin pressure
10×
The cost per keyword for a 10-page deep pull compared to a single page, when each page is billed independently at a flat rate. Most SERP APIs do exactly this.
Silent data loss
1 in 6
A partial SERP failure — one page of ten coming back empty — looks like a complete dataset until you check rank position distributions. Dashboards stay green while the data degrades.
Engineering overhead
2–4 wks
The typical sprint cost every time pagination logic, proxy configuration, or retry behaviour needs to be rebuilt after an enforcement change. It is not a one-time cost.
Strategic blindspot
~30%
Of tracked queries now return an AI Overview. Teams without structured capture of these blocks are monitoring what used to matter, not what their customers actually see.
Try Zyte APITalk to our Data Experts
See the Schema

The request you send and the data that comes back

Pick the standard schema or a custom one mapped to your model, and read the response as a table or JSON.

REQUEST
POST https://api.zyte.com/v1/extract

{

  "url": "https://example-search.com/search?q=best+enterprise+SEO+platform",

  "serp": true

}
RESPONSE
Standard Search & SERP schema
Field
Type
Example
url
string
https://example-search.com/search?q=best+enterprise+SEO+platform[https://example-search.com/search?q=best+enterprise+SEO+platform]
keyword
string
best enterprise SEO platform
geo
object
{ … }
country
string
US
language
string
en
collectedAt
string
2026-07-02T08:00:00Z
pageDepth
integer
10
resultsTotal
integer
100
serpFeatures
array
["aiOverview", "organicResults"]
aiOverview
object
{ ... }
captured
string
captured
organicResults
array
[{...}]
rankPosition
number
1
url
string
seoplatform.com/enterprise [http://seoplatform.com/enterprise]
title
string
Best Enterprise SEO Platform 2026
snippet
string
Compare the top enterprise SEO platforms by features, pricing, and support.
Reviews

What our users say

I have been working with Zyte's team for the last few months, and their team is fantastic. I appreciate their development speed and quality, and they run a very robust platform, producing very satisfactory results. I love the ease of the initial setup with Zyte, as they took care of all the development, and we only needed to communicate what data we needed and set up the necessary processes on our end.

DP
David P.

The best thing about Zyte is their flexibility. There was a certain website that we needed to crawl. They were able to adapt their code to increase its capabilties. Zyte has a good amount of features and different offerings. Customer support is very good, technical and business. Easy to use and integrate in our code base.

TL
Thierry L.

I find Zyte really easy to use, and it was very easy and pretty self explanatory to set up. We were able to integrate the information we pull into our website to directly compare pricing with other leading companies. Zyte allows us to easily set up the site scrapes exactly how we need it, and it does exactly what we need it to do, always providing a way to make that happen.

BB
Brandon B.

I like Zyte's pricing model which offers a pay-as-you-go level. You only pay when Zyte successfully retrieves the HTML page, and if Zyte has an error, you don't pay. Their support is responsive and their platform is accessible via API. Additionally, the integration with the Python library was simpler.

FP
Florian P.

I love Zyte's after-sales service. Additionally, the setup process was straightforward because I had the support of the technical team and the Customer Success Manager at all times. I greatly appreciate that Zyte delivers value, which is very valuable to us, especially considering the complex programming work and the continuous changes on competitors' web pages.

JS
Jose S. (daas)

Zyte help build our web scrapers and ensure we get structured data regularly. They are the most personable supplier I've worked with, and it's great to meet the people behind it regularly. Communication is straightforward. If there's an issue, I can sit with them and find a solution collaboratively.

AM
A. M.

I love the simplicity of integrating Zyte into existing Python code. The API is very convenient because it automatically handles retries in case of failure and allows for parallelizing requests. The initial installation of Zyte was very easy.

YM
Yann M.

We used Zyte because it's a reliable way to scrape large amounts of data without risking being blocked by websites' security systems. I really appreciate its 'do once and forget about it' approach; we set everything up once, and it continued to work for years without a hitch. The reliability is impressive - it always works, and for a scraping tool, that's very important.

AN
Alexandr N.

Frequently asked questions

Does Zyte's /search endpoint replace what bulk retrieval parameters used to do?

Yes, and more precisely. Zyte's coordinated pagination returns up to 100 organic results from a single API call — one request, ten pages fetched and assembled internally. Pages 2–10 share the session and proxy infrastructure established on page 1, so they cost less to run and less to buy. You get the same depth you had before at a materially better unit cost than ten independent requests would produce.

Do you handle JavaScript-rendered SERPs, including AI Overviews?

Yes. Zyte uses browser rendering by default for SERP collection. AI Overview content is returned inside the response HTML for every query where it is served — your pipeline sees the full page, not the pre-render shell.

What formats and delivery methods do you support?

Zyte delivers data in JSONL, Parquet, CSV, and custom formats, via S3, SFTP, API, or direct warehouse integration. Format and delivery cadence are scoped per project.

How fresh can SERP data be?

Rank data is typically delivered daily or on-demand; freshness is scoped per use case based on how often the relevant SERPs actually change. Real-time and event-triggered collection is available where needed.

How do you handle layout changes that break extraction?

Zyte validates every collection run against your agreed schema. Layout changes trigger automatic detection and escalation before gaps reach your data.

What about anti-bot measures and blocks?

When a request is blocked, Zyte automatically re-routes and retries with a different proxy, session, or rendering approach. Persistent blocks escalate to the team running your pipeline rather than silently failing.

What formats and delivery methods do you support?

Structured JSON via the API, with delivery to your preferred destination. Custom schema mapping is available for teams with existing data models.

How long does setup take?

Self-serve access starts immediately via Zyte API. Enterprise agreements with custom schema, volume pricing, and dedicated support are typically scoped and live within a few weeks.

Can I see a sample before committing?

Yes. Talk to a data specialist to request a sample run on the keywords and markets you care about.

Services

Zyte Data

Fully managed web data extraction, delivered to your spec.

Explore Zyte Data

Web Scraping API

Zyte API

Scrape any website at scale with automatic proxy rotation and ban handling.

Sign Up

Developers

Zyte Developers

Docs, tools, and a community to help you build and scale scrapers.

Join Us
    • Zyte API
    • Ban Handling
    • AI Extraction
    • SERP
    • Enterprise
    • Scrapy Cloud
    • Agentic Web Data
    • Pricing
    • Product & E-commerce
    • Data for AI
    • Job Posting
    • Real Estate
    • News & Articles
    • Search
    • Social Media
    • Blog
    • Learn
    • Case Studies
    • Webinars
    • White Papers
    • Join our community
    • Join our Affiliate Program
    • Documentation
    • Meet Zyte
    • Contact us
    • Jobs
    • Support
    • Terms and Policies
    • Trust Center
    • Do not sell
    • Cookie settings
    • Web Data Compliance
    • Open Source
    • What is Web Scraping
    • Web Scraping in Python: Ultimate Guide
    • Stop getting blocked, start scraping
  • Logo EWDCILogo Most Loved WorkplaceLogo Job TogetherISO 27001 SealMedal Leader Europe Winter 2025Fastest Implementation Winter 2025Logo Leader Winter 2025Grid Leader Spring 2025Grid Leader Summer 2025Leader Fall 2025Leader Winter 2026
    XFacebookInstagramYouTubeLinkedInDiscord

    © Zyte Group Limited 2026