Developer Advocate

Ayan Pahwa

Ayan Pahwa is developer advocate at Zyte.
When not creating new projects and speaking about them at developer conferences, you can find Ayan scaling and maintaining his self-hosted homelab. Ayan is also an open source contributor and actively participates in building and growing inclusive communities.’

Subscribe to our Blog

The latest from Ayan Pahwa

Web scraping on an iPhone? Yes, really!

When you can scrape the web by API, a world of possibility opens up. Yes, you can extract live web data using iOS Shortcuts.

Automate deployment of your web scraper on VPS with Ubuntu 24.04 cloud-init

Your VPS is ready, but now you need to work through the same sequence you have run a dozen times before: apt update, apt install python3-pip, pip install scrapy, playwright install chromium, the Chromium dependency list that never installs cleanly on the first try, Redis, possibly Postgres, whatever else this particular project needs.

Web scraping on 22 KB of RAM: Fitting the world on an ESP8266 microcontroller

Data-gathering doesn’t have to be memory-intensive. You can fit the world’s weather on a 9cm-square board, when you move the work to a web scraping API.

Flatcar Linux for web scrapers: deploy immutable containers with just one config file

the next time you spin up a VPS to give it a persistent home, you spend the better part of an afternoon rebuilding from memory: installing Scrapy, wiring up Redis, configuring the systemd units, getting Playwright's Chromium dependencies in the right state. Here's a tool to help

My agentic coding setup: Claude Code, multi-agent orchestration, and how I actually work

Ayan's 4 agent team, using Claude's /goal, and the models and coding agents he uses to code effectively.

OpenClaw and Claude helped me buy the perfect sneakers using Zyte API

Quickly compare e-commerce products across any site with an agent, a skill and an AI-powered web scraping API.

Giving spidey-senses to your web scraping spiders using Spidermon

Learn how Spidermon helps you monitor web scraping data quality in real time. Validate items, track field coverage, and get alerts before bad data impacts your pipeline.

Code is cheap, show me the talk: How copilots are re-engineering developers

Mastery of computer code used to be an engineer’s differentiator. Thanks to AI assistants, code is now the commodity, sensibility is the real premium.

Build your own MCP server: LLMs meets web data with Zyte API

Learn how to build your own Model Context Protocol (MCP) server to connect LLMs with real-time web data using Zyte API, FastMCP, and the Docker MCP toolkit.

Brewing a bot: RAG and web data fuel the perfect coffee recommendation

Learn how to build a real-time AI chatbot using RAG, web scraping, Zyte API, LangChain, and OpenAI. Scrape JavaScript-heavy websites, store data in a vector database, and generate accurate answers from fresh web data.

Scraping Swiss Army Knife: My personal fix for web setup fatigue using Docker, Scrapy and Zyte

Tired of repeating web scraping setup? Learn how a multi-arch Docker container with Scrapy, Zyte, Requests, and Pandas speeds up exploration and debugging.

How I trade gold using e-ink, live data and an old Raspberry Pi

Track real-world gold and silver retail prices automatically using Zyte API, Python, and a Raspberry Pi with an e-ink display. Learn how to scrape rendered HTML, parse prices, and build an always-on trading dashboard.