PINGDOM_CHECK

#ExtractSummit2026 The world's largest web scraping conference returns. Austin Oct 7–8 · Dublin Nov 10–11

Register now
Data Services
Login
Try Zyte APIContact Sales
  • Unblocking and Extraction

    Zyte API

    The ultimate API for web scraping. Avoid website bans and access a headless browser or AI Parsing

    Ban Handling

    Headless Browser

    AI Extraction

    SERP

    Enterprise

    DocumentationSupport

    Hosting and Deployment

    Scrapy Cloud

    Run, monitor, and control your Scrapy spiders however you want to.

    Coding Agent Add-Ons

    Agentic Web Data

    Plugins that give coding agents the context to build production Scrapy projects. Starts with Claude Code.

  • Data Services
  • Zyte API

    Zyte Data

    Scrapy Cloud

  • Browse

    • BlogArticles, podcasts, videos
    • Case studiesCustomer outcomes
    • White papersIn-depth reports
    • DocumentationGuides & API reference
    • EventsConferences, webinars, recordings

    Subscribe

    • NewsletterSwiftly delivered
    • Join our community2,000+ web scraping engineers
  • Product and E-commerce

    From e-commerce and online marketplaces

    Data for AI

    Collect and structure web data to feed AI

    Job Posting

    From job boards and recruitment websites

    Real Estate

    From Listings portals and specialist websites

    News and Article

    From online publishers and news websites

    Search

    Search engine results page data (SERP)

    Social Media

    From social media platforms online

  • Meet Zyte

    Our story, people and values

    Contact us

    Get in touch

    Support

    Knowledge base and raise support tickets

    Terms and Policies

    Accept our terms and policies

    Open Source

    Our open source projects and contributions

    Web Data Compliance

    Guidelines and resources for compliant web data collection

    Affiliate Program

    Join Zyte’s affiliate program and start earning commissions today

    Join the team building the future of web data
    We're Hiring
    Trust Center
    Security, compliance & certifications
Login
Try Zyte APIContact Sales
All articles
AI75, 75 articles
Data quality15, 15 articles
Developer interest60, 60 articles
Integration3, 3 articles
Open-source50, 50 articles
Proxies35, 35 articles
Scraping practice35, 35 articles
Scraping strategy48, 48 articles
Search results4, 4 articles
Web data75, 75 articles
Web scraping APIs49, 49 articles
Scrapy47, 47 articles
Scrapy Cloud26, 26 articles
Web Scraping Copilot11, 11 articles
Zyte API71, 71 articles
AI & Machine Learning3, 3 articles
Automotive3, 3 articles
E-commerce & retail35, 35 articles
Entertainment & Streaming2, 2 articles
Financial Services8, 8 articles
Government2, 2 articles
Market Research & Intelligence7, 7 articles
Media & publishing11, 11 articles
Real Estate2, 2 articles
Recruitment & HR3, 3 articles
Transportation & Logistics2, 2 articles
Travel & hospitality3, 3 articles
iPaaS2, 2 articles
Large language model29, 29 articles
MCP3, 3 articles
Python110, 110 articles
Scraping at Scale7, 7 articles
Scraping Fundamentals11, 11 articles
Web Scraping Industry Report20, 20 articles

Appearance

Discord Community
BlogHow ToRunning Playwright at scale: connecting to the Zyte CDP browser
ArticleTutorial / How-toHow ToIntegration

Running Playwright at scale: connecting to the Zyte CDP browser

We've released our CDP Browser, ideal for those with existing Playwright scripts looking for better access, as well as developers who need granular control over a browser, but don't want the hassle of running locally.

John Rooney · Developer Engagement Manager

September 7, 2026

Running Playwright at scale: connecting to the Zyte CDP browser

We've just released the Zyte CDP browser, so this post covers what it is, what it's good for, and, just as important, when you shouldn't use it. If you've got existing Playwright or Puppeteer scripts and you're tired of babysitting browser infrastructure, this is for you.

What is CDP?

CDP is the Chrome DevTools Protocol, a way of connecting to and controlling a running browser, and it's neither new nor exotic. Chrome DevTools itself uses this protocol every time you inspect network requests or run JavaScript in the console on an open page.

As web scrapers, we rarely touch CDP directly - instead we use Playwright or Puppeteer as a wrapper around the underlying CDP commands. I'll stick to Playwright here, and through it we can launch browsers, connect to them, run actions on pages, and everything else the job needs.

The problem with local browsers

The standard workflow looks like this: your script downloads a Chrome binary, launches it, and connects to it. This works fine on your laptop, but it falls apart the moment you need scale.

Browsers are resource-heavy, they eat memory, and running them in any real volume is an infrastructure project in its own right. You end up managing containers, memory limits, zombie processes, and crash recovery instead of writing scrapers, and if your scrapers already run in Docker or Kubernetes, the browser pods are the biggest and least reliable things in the cluster.

The fix is to run the browsers somewhere else, connecting to a remote browser on separate infrastructure while keeping your scripts exactly as they are. That's what the Zyte CDP browser is, browsers running on our infrastructure with your Playwright scripts driving them over CDP.

You also get our web scraping technology bundled in, so ban solving and fingerprinting come with the browser and you're not solving those problems yourself either.

An important distinction before you start

If all you need is rendered HTML and you don't have something already build, don't use this, use browserHtml with Zyte API instead. It's simpler and it's the right tool for that job.

The CDP browser makes sense when you have a Playwright script that performs many different actions, needs fine control over the browser, or runs through a flow where state has to carry from one step to the next. Zyte API actions can already handle some of this, but they're much more limited, and direct CDP access removes those limits.

Why we built it

This didn't come out of a product roadmap meeting, it came out of a run of customer conversations where the same few problems kept coming up.

Code nobody wanted to rewrite. One team scraping real estate listings had built their whole dynamic pipeline on Playwright and Puppeteer, and moving to Zyte API meant translating every one of those scripts into API calls or the Actions model. They looked at the size of that job and didn't want to do it, which is fair enough. With a CDP endpoint the scripts stay as they are, the connection line changes and nothing else does.

A second provider. A job-market data startup was already running on another vendor's hosted browser over CDP, and it was failing outright on certain drag-and-drop challenges. They didn't want to switch, they wanted a fallback for when the first option broke, and since the interface is CDP either way, swapping providers is a config change. They also wanted the browser to work out which proxy tier a site needs on its own, datacenter by default and residential only when the site forces it, which is how Zyte API already handles bans and the CDP browser inherits it.

Their sessions never needed to live longer than five minutes, by the way, they just wanted to connect, extract, and disconnect. Not every CDP use case is a long-running one.

Smart Proxy Manager users. SPM supported browser automation, and Zyte API's proxy mode didn't cover everything those users relied on, so with SPM being retired in 2026, people running Puppeteer or Playwright through it needed somewhere to go that didn't mean a rewrite. The CDP browser is that.

Fingerprinting. Websites look at hardware footprint, mouse movement, and keyboard timing, and a stock headless script advertises itself as a bot and gets banned on the first request. Running a customised browser on our side, with our fingerprint and IP handling behind it, takes that maintenance off your plate. I'm not going to claim it gets you everything, but it does mean the browser isn't the weakest link in your stack anymore.

Connecting Playwright to the Zyte browser

Here's the simplest possible example, where you connect with a CDP connection string and authenticate with your Zyte API key:

That's it. connect_over_cdp is a standard Playwright method, and if you check its docstring it describes connecting via the Chrome DevTools Protocol, which is exactly what's happening here. Puppeteer's puppeteer.connect takes the same kind of websocket endpoint if that's your tool.

If you already have a Playwright script, this connection block is the only part you change to move from a local or other remote browser onto our infrastructure, and everything after the connection stays the same.

Interactions are where it gets useful

Rendering a page over CDP works, but the value really shows up when your script interacts with pages, because page.click, page.fill, waiting for selectors, anything Playwright exposes, all of it runs against the remote browser:

The script connects to our infrastructure, runs the whole sequence, and returns the data, and the more complicated your actions, the more this pays off compared to expressing everything through API action definitions.

Blocking images to cut data transfer

One thing I do a lot is block images, since they're usually dead weight in a scraping job and blocking them saves a surprising amount of data transfer and loading time:

It's one line, and worth doing by default unless you need the images.

Screenshots

Playwright's page.screenshot runs against the remote browser like everything else:

Raw CDP commands

Playwright covers most needs, but a direct CDP session gives you access to browser internals that Playwright doesn't wrap, such as overriding the browser's timezone:

You can change a lot about the browser this way, and this level of control is the reason to have a direct CDP connection at all, since a request/response API can't give it to you.

Using it with Scrapy

If you're working in Scrapy, the scrapy-playwright extension normally downloads and launches a local browser, which is exactly the pattern I warned about at the start, so point it at our infrastructure with the CDP URL instead:

If you have existing Playwright scripts doing complicated actions and you want off local browsers, in a lot of cases it really is this straightforward, you add the CDP URL and the work moves to a remote browser.

Giving AI agents browser access

Claude wrote all the example code in this post's companion video, and that's the point here, because if you give an agent the CDP connection string and an API key, it can write its own Playwright scripts for a task and run them against a remote browser.

That means the machine your agent runs on can be tiny, since it doesn't need to run browsers itself when we handle that off-site. The agent writes code, the browser lives on our infrastructure, and you skip the whole problem of provisioning browser-capable compute for agent workloads.

When to use it, when not to

Use the CDP browser when:

  • You have existing Playwright scripts with many different actions
  • You need fine control over the browser, including raw CDP commands
  • Your flow spans several steps and the site has to remember state between them, such as a solved challenge, a login, or a filled form
  • You're moving off Smart Proxy Manager or another hosted browser and don't want to rewrite anything
  • You want agents to drive real browsers without hosting them

Skip it when you just need rendered HTML, because Zyte API with browserHtml is the better option there.

Links to pricing, access, and the community are below, and if you build something with this, come tell us what you used it for.

CDP Homepage

CDP Docs

Join our community

Try Zyte API

Build your first scraper in minutes

Free trial, no credit card. From a single request to production in an afternoon.

Get started
How ToIntegration

John Rooney

Developer Engagement Manager

John is the Developer Engagement Manager at Zyte, working closely with the community, creating content and helping developers learn web scraping, Zyte products an much more. He has spoken at Extract Summit's and also creates the workshop's for the events.

  • X (Twitter)
  • LinkedIn
More from this author

In this article

  • What is CDP?
  • The problem with local browsers
  • An important distinction before you start
  • Why we built it
  • Connecting Playwright to the Zyte browser
  • Interactions are where it gets useful
  • Blocking images to cut data transfer
  • Screenshots
  • Raw CDP commands
  • Using it with Scrapy
  • Giving AI agents browser access
  • When to use it, when not to

Follow

Get the latest

Zyte and the data web in your inbox — or wherever you already are.

Subscribe

Or follow elsewhere

Continue reading

Teaching AI to scrape like a pro: how we measure LLMs’ data quality
How To

Teaching AI to scrape like a pro: how we measure LLMs’ data quality

AI-enabled code editors can now conjure scraping code on command. But is it any good? Here’s how Zyte re-engineered LLMs with Web Scraping Copilot to drive best-in-class output.

Theresia Tanzil·10 min·February 23, 2026
Analyze web data quickly with Jupyter Notebooks and Zyte API
How To

Analyze web data quickly with Jupyter Notebooks and Zyte API

With AI Scraping in Zyte API, you can pull data from any e-commerce website straight into your Jupyter notebooks.

Neha Setia Nagpal·2 mins·December 13, 2024
Overcoming web scraping challenges of Puppeteer and Playwright
How To

Overcoming web scraping challenges of Puppeteer and Playwright

Discover the challenges of scaling web scraping with Playwright & Puppeteer, from browser farm management to IP rotation and anti-scraping tactics.

Neha Setia Nagpal·1 mins·December 5, 2024
Inside Zyte's System Design Process: How We Build Scalable, Reliable Solutions
How To

Inside Zyte's System Design Process: How We Build Scalable, Reliable Solutions

Explore Zyte’s approach to building scalable and reliable systems through PRDs, technical requirements, solution evaluation, and real-world design insights.

Alexander Sibiryakov·1 mins·December 19, 2024
Leveraging Web Scraping and Big Data: The New Frontier in Optimized Delivery Solutions
How To

Leveraging Web Scraping and Big Data: The New Frontier in Optimized Delivery Solutions

Big Data Delivery isn’t just about moving information around—it’s about making it work for you, helping businesses spot trends, predict what’s next, and stay ahead in a cutthroat market.

Karlo Jedud·10 mins·January 2, 2025
AI Web Scraping as the Future of Scalable Data Collection
How To

AI Web Scraping as the Future of Scalable Data Collection

AI-powered web scraping is transforming data collection by making it faster, smarter, and highly scalable. Learn how it overcomes traditional scraping challenges and unlocks new opportunities for businesses across industries.

Karlo Jedud·5 mins·September 4, 2025

The Community · Newsletter

The best of Zyte and the data web, in your inbox.

One curated edition — new articles, product updates, and the stories shaping the data web. No noise.

Services

Zyte Data

Fully managed web data extraction, delivered to your spec.

Explore Zyte Data

Web Scraping API

Zyte API

Scrape any website at scale with automatic proxy rotation and ban handling.

Sign Up

Developers

Zyte Developers

Docs, tools, and a community to help you build and scale scrapers.

Join Us
    • Zyte API
    • Ban Handling
    • AI Extraction
    • SERP
    • Enterprise
    • Scrapy Cloud
    • Agentic Web Data
    • Pricing
    • Product & E-commerce
    • Data for AI
    • Job Posting
    • Real Estate
    • News & Articles
    • Search
    • Social Media
    • Blog
    • Learn
    • Case Studies
    • Webinars
    • White Papers
    • Join our community
    • Join our Affiliate Program
    • Documentation
1import asyncio
2from base64 import b64encode
3from playwright.async_api import async_playwright
4
5ZYTE_API_KEY = "your-api-key"
6
7CDP_URL = "wss://browser.zyte.com"  # verify against docs
8AUTH = b64encode(f"{ZYTE_API_KEY}:".encode()).decode()
9
10async def main():
11    async with async_playwright() as p:
12        browser = await p.chromium.connect_over_cdp(
13            CDP_URL,
14            headers={"Authorization": f"Basic {AUTH}"},
15        )
16        page = await browser.new_page()
17        await page.goto("https://example.com")
18        print(await page.title())
19        await browser.close()
20
21asyncio.run(main())
Copy
1page = await browser.new_page()
2await page.goto("https://example.com/search")
3
4await page.fill("#search-input", "web scraping")
5await page.click("#search-button")
6await page.wait_for_selector(".results")
7
8data = await page.locator(".results .item").all_text_contents()
Copy
1await page.route(
2    "**/*.{png,jpg,jpeg,gif,webp,svg}",
3    lambda route: route.abort(),
4)
Copy
1screenshot = await page.screenshot(full_page=True)
2with open("page.png", "wb") as f:
3    f.write(screenshot)
Copy
1cdp = await page.context.new_cdp_session(page)
2await cdp.send("Emulation.setTimezoneOverride", {
3    "timezoneId": "America/New_York",
4})
Copy
1# settings.py
2PLAYWRIGHT_CDP_URL = "wss://browser.zyte.com"  # verify against docs
3
4DOWNLOAD_HANDLERS = {
5    "http": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
6    "https": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
7}
Copy
    • Meet Zyte
    • Contact us
    • Jobs
    • Support
    • Terms and Policies
    • Trust Center
    • Do not sell
    • Cookie settings
    • Web Data Compliance
    • Open Source
    • What is Web Scraping
    • Web Scraping in Python: Ultimate Guide
    • Stop getting blocked, start scraping
  • Logo EWDCILogo Most Loved WorkplaceLogo Job TogetherISO 27001 SealMedal Leader Europe Winter 2025Fastest Implementation Winter 2025Logo Leader Winter 2025Grid Leader Spring 2025Grid Leader Summer 2025Leader Fall 2025Leader Winter 2026

    © Zyte Group Limited 2026
    XFacebookInstagramYouTubeLinkedInDiscord