PINGDOM_CHECK

#ExtractSummit2026 The world's largest web scraping conference returns. Austin Oct 7–8 · Dublin Nov 10–11.

Register now
Data Services
Pricing
Login
Try Zyte APIContact Sales
  • Unblocking and Extraction

    Zyte API

    The ultimate API for web scraping. Avoid website bans and access a headless browser or AI Parsing

    Ban Handling

    Headless Browser

    AI Extraction

    SERP

    Enterprise

    DocumentationSupport

    Hosting and Deployment

    Scrapy Cloud

    Run, monitor, and control your Scrapy spiders however you want to.

    Coding Agent Add-Ons

    Agentic Web Data

    Plugins that give coding agents the context to build production Scrapy projects. Starts with Claude Code.

  • Data Services
  • Pricing
  • Browse

    • BlogArticles, podcasts, videos
    • Case studiesCustomer outcomes
    • White papersIn-depth reports
    • DocumentationGuides & API reference
    • EventsConferences, webinars, recordings

    Subscribe

    • NewsletterSwiftly delivered
    • Discord communityExtract Data community
  • Product and E-commerce

    From e-commerce and online marketplaces

    Data for AI

    Collect and structure web data to feed AI

    Job Posting

    From job boards and recruitment websites

    Real Estate

    From Listings portals and specialist websites

    News and Article

    From online publishers and news websites

    Search

    Search engine results page data (SERP)

    Social Media

    From social media platforms online

  • Meet Zyte

    Our story, people and values

    Contact us

    Get in touch

    Support

    Knowledge base and raise support tickets

    Terms and Policies

    Accept our terms and policies

    Open Source

    Our open source projects and contributions

    Web Data Compliance

    Guidelines and resources for compliant web data collection

    Join the team building the future of web data
    We're Hiring
    Trust Center
    Security, compliance & certifications
Login
Try Zyte APIContact Sales
All articles
AI66, 66 articles
AI-assisted data extraction31, 31 articles
Data gathering for AI4, 4 articles
Large Language Models (LLMs)14, 14 articles
Tool-assisted coding4, 4 articles
Data quality13, 13 articles
Developer interest57, 57 articles
Integration2, 2 articles
Open-source41, 41 articles
Proxies29, 29 articles
Scraping practice19, 19 articles
Scraping strategy29, 29 articles
Web data60, 60 articles
Web scraping APIs36, 36 articles
Scrapy47, 47 articles
Scrapy Cloud14, 14 articles
Web Scraping Copilot11, 11 articles
Zyte API57, 57 articles
AI & Machine Learning3, 3 articles
Automotive2, 2 articles
E-commerce & retail27, 27 articles
Entertainment & Streaming2, 2 articles
Financial Services8, 8 articles
Government2, 2 articles
Market Research & Intelligence3, 3 articles
Media & publishing8, 8 articles
Real Estate2, 2 articles
Recruitment & HR3, 3 articles
Transportation & Logistics2, 2 articles
Travel & hospitality2, 2 articles
Extract Summit25, 25 articles
PyCon1, 1 articles
iPaaS2, 2 articles
Large language model24, 24 articles
MCP3, 3 articles
Python88, 88 articles
Web Scraping Industry Report14, 14 articles

Appearance

Discord Community

AI

Articles from the Zyte blog about AI.

Agentic Web Data toolbelt
AI-assisted data extraction

New Zyte add-ons: Agent Skills, Codex, GitHub and more updates

A host of additions to Zyte’s agentic scraping toolbelt helps developers go from prompt to working spider code and beyond.

Valter Sciarrillo·10 min read·July 22, 2026
harness-engineering-3
AI

Harness Engineering #3- Headless mode: the minimal agent harness

What's the smallest harness that still works? Turns out it's already sitting inside almost every coding agent you have installed — headless mode: same loop, tools, and reasoning as the interactive agent, minus the human in the chair. We point it at a web page and pull clean, structured data out the other end in about ten lines.

Ayan Pahwa·12 min read·July 13, 2026
An automatic Claude writer device, writing code.
AI-assisted data extraction

Claude Fable 5 is the new best model for writing scrapers

We ran nine models, including the new GPT-5.6, through our Zyte Scraping Code Benchmark. Claude Fable 5 puts Sol and others in the shade - but it’s pricey, and the best extraction code still depends on the best infrastructure.

Konstantin Lopukhin·8 min read·July 10, 2026
gpt5-6-cover
AI

GPT-5.6, Fable 5, and GLM-5.2 entered a bar “crawl” and got hit by The Rate Limit

When OpenAI shipped GPT-5.6 and Anthropic's Fable 5 was sitting at the top of the price list, my question was not "which one wins a leaderboard." It was the one I actually pay for: for the scraping I do, how much model do I need to buy?

Ayan Pahwa·16 min read·July 10, 2026
skills-are-software
Open-source

Treat your AI skills like software, starting with evals

Most AI skills are never tested — and it shows. Here's how Zyte evaluates scraping skills like real software, catching failures demos miss.

Neha Setia Nagpal·14 min read·July 8, 2026
Harness Engineering, part 2: harnessing a data extraction agent
Large Language Models (LLMs)

Harness Engineering, part 2: harnessing a data extraction agent

Point it at a website, tell it which fields you want, get back clean structured records. That's the agent we're designing in this post — and the interesting part isn't the model, it's the harness decisions that make it actually reliable at scale.

Ayan Pahwa·8 min read·July 1, 2026
Harness Engineering, part 1: what is an agent harness and why it matters
Large Language Models (LLMs)

Harness Engineering, part 1: what is an agent harness and why it matters

Same model, same weights, zero retraining — LangChain changed nothing but the scaffolding around a coding model and jumped it from 30th place to the top five on a benchmark. That scaffolding has a name: the harness. And it's the part you actually control.

Ayan Pahwa·8 min read·June 29, 2026
The harness matters more than the model - Podcast EP07
AI-assisted data extraction

The harness matters more than the model - Podcast EP07

"The model is the engine — but the harness is everything else." In Episode 7, we dig into why the infrastructure layer around your AI model matters more than the model itself, rank the best models available right now, and ask whether the open-weighted revolution is about to make frontier subscriptions obsolete.

John Rooney·11 min read·June 27, 2026
C N T 1216 Main
AI

The best agent skill is the one that says the least

More instruction, worse output. Zyte's head of R&D on why telling your agent exactly what to do can blind it to the obvious answer.

Neha Setia Nagpal·14 min read·June 23, 2026
Why I'm adding GLM-5.2 to my agentic coding arsenal
Large Language Models (LLMs)

Why I'm adding GLM-5.2 to my agentic coding arsenal

Is GLM-5.2 really closing the gap to Anthropic - and at just a fraction of the cost - or is it just more AI hype? I think so, and let me show you why.

Ayan Pahwa·12 min read·June 19, 2026
My personal agent setup: the architecture that runs my household and my DevRel work
AI

My personal agent setup: the architecture that runs my household and my DevRel work

"Four people, four diets, two work schedules, and a baby who answers to nobody. That's what finally made me build a personal agent." A walkthrough of the actual architecture I run to hold my household and my DevRel work together — profiles, skills, memory, and the web-data layer that makes it all reach the live web.

Neha Setia Nagpal·11 min read·June 16, 2026
What's becoming of web scraping developers in the age of AI agents?
AI-assisted data extraction

What's becoming of web scraping developers in the age of AI agents?

AI agents can generate code, suggest selectors, and draft crawl logic. What they can't do is design the system that decides when to stop, what to trust, and how to recover when the web pushes back. That job still belongs to a human.

Neha Setia Nagpal·14 min read·June 9, 2026
12345

Services

Zyte Data

Coding tools & hacks straight to your inbox. Bi-weekly dosage of all things code.

Talk to us

Web Scraping API

Zyte API

Coding tools & hacks straight to your inbox. Bi-weekly dosage of all things code.

Sign Up

Developers

Zyte Developers

Coding tools & hacks straight to your inbox. Bi-weekly dosage of all things code.

Join Us
    • Zyte API
    • Ban Handling
    • AI Extraction
    • SERP
    • Enterprise
    • Scrapy Cloud
    • Agentic Web Data
    • Pricing
    • Product & E-commerce
    • Data for AI
    • Job Posting
    • Real Estate
    • News & Articles
    • Search
    • Social Media
    • Blog
    • Learn
    • Case Studies
    • Webinars
    • White Papers
    • Join our community
    • Documentation
    • Meet Zyte
    • Contact us
    • Jobs
    • Support
    • Terms and Policies
    • Trust Center
    • Do not sell
    • Cookie settings
    • Web Data Compliance
    • Open Source
    • What is Web Scraping
    • Web Scraping in Python: Ultimate Guide
    • Stop getting blocked, start scraping
  • EWDCI logoMost loved workplace certificateZyte rewardISO 27001 iconG2 rewardG2 rewardG2 reward
    XFacebookInstagramYouTubeLinkedInDiscord

    © Zyte Group Limited 2026