PINGDOM_CHECK

End year review: The most important things in 2019 for us

2019 has been a monumental year for us. With the wealth of experience and knowledge that we collected in 2019, we hope to make 2020 even better for you!

Attila Toth

4 min read ·

End year review: The most important things in 2019 for us

Looking back at 2019

2019 was an exciting year for Zyte . We created things we have never created before and did things nobody in our industry had ever done before. Let’s revisit what happened in 2019!

The first-ever web data Extraction Summit

We organized the first industry event focused on Web Data Extraction in Dublin, Ireland. With more than 140 attendees, 16 speakers covering topics from technical deep dives to business use cases, 12 presentations, a customer panel discussion, and unlimited Guinness. It was an absolute success.

It was an awesome event. We loved it and the people who attended loved it, with 94% of attendees giving it an overall rating of Excellent/Good. So we are going to do it again! We’re going to make sure Extract Summit 2020 is going to be even bigger and better. Watch out for our call for speakers which we will launch in February.

Shaping the future of web scraping

We launched Automatic Extraction API our AI-enabled automatic web scraping Product. At the core of the Automatic Extraction is an AI-enabled data extraction engine able to extract data from a web page without the need to design custom code. Through the use of deep learning, computer vision, and Smart Proxy Manager , Zyte's advanced proxy management solution, the data engine is able to automatically identify common items on product and article web pages and extract them without the need to develop and maintain extraction rules for each site.

Other verticals will be added to the API in the coming months such as job postings, real estate, automotive, product reviews, blogs and discussions, to name a few, will be among the verticals that we plan to roll out in 2020. It’s going to be exciting! We are always looking for Beta customers to join so get in touch if you’re interested in any of the verticals mentioned above.

Zyte Smart Proxy Manager

2019 was a big year for Zyte Smart Proxy Manager . We made huge advancements on the backend side of things, improving reliability and robustness. The success rate Zyte Smart Proxy Manager provides is unmatched in the industry but still, we found ways to make it stronger and more powerful to provide our customers with the best web scraping proxy solution in the market.

Looking at the type of successful requests our Zyte Smart Proxy Manager customers rely on us for, a huge portion of the requests are eCommerce websites followed by app stores, social networks, and search engines. E-Commerce web data extraction for price intelligence was one of the top solutions requested by customers in 2019.

Open source

Open source is something we believe in and put a lot of effort into last year. In 2019, we open-sourced two libraries to help Scrapy developers scrape the web. Spidermon is a Scrapy extension to monitor your spiders, get statistics, notifications and validate data. Arche is a tool for verifying scraped data. These two repositories together got 264 stars on GitHub.

EY Entrepreneur of the year

Shane Evans our CEO made us all extremely proud when he was nominated for EY Entrepreneur of the year, Ireland. This is a unique global program that recognizes entrepreneurial achievement among individuals and companies that demonstrate vision, leadership, and success - and work to improve the quality of life in their communities, countries, and around the world. You can read about his story in an interview with the Irish Times.

{% video_player "embed_player" overrideable=False, type='scriptV4', hide_playlist=True, viral_sharing=False, embed_button=False, width='1920', height='1080', player_id='25059995550', style='' %}

Events

Other than holding our own event, last year we also visited Battlefin, the Internet Retailing Expo (IRX), and the AI and Big Data Expo, in London. It is always great to spread the word of the power of web scraping and show people what it can do for them.

gree aero tours

Remote working and team meetups

We love remote working. It’s in our DNA. In 2019 we got involved with initiatives such as GrowRemote and RunningRemote to help educate others on how to run a successful remote-first company. Our Head of Marketing, Marie Moynihan was invited to speak on national radio on The Home Show with Sinead Ryan to chat about what it’s like to work remotely. It’s a great podcast episode, you can listen to it here.

Team meetups

gree aero tours (1)As a fully remote company, it’s important to find the time and place to get together and meet each other in person. This year we decided to run regional team meetups instead of a larger company retreat. The meetups were a great success and allowed functional teams to co-working time together and the ability to get that all-important face-time together as a team and of course have some fun. Meetups were held in Dublin, Montevideo, Madrid, and Kuala Lumpur.

Looking forward to 2020

We have ambitious goals for the year and this year is going to be exciting!. We are seeing increasing demand plus new use cases for web data and are excited to continue to provide solutions to our current and future customers. We will also be celebrating our 10 year anniversary!

If you would like to be part of our journey, have a look at our open roles. If you want to know how web extracted data can help your business, contact us for a free consultation. In 2020, let’s grow together

Try Zyte API

Build your first scraper in minutes

Free trial, no credit card. From a single request to production in an afternoon.

Get started

Continue reading

The harness matters more than the model - Podcast EP07
AI-assisted data extraction

The harness matters more than the model - Podcast EP07

"The model is the engine — but the harness is everything else." In Episode 7, we dig into why the infrastructure layer around your AI model matters more than the model itself, rank the best models available right now, and ask whether the open-weighted revolution is about to make frontier subscriptions obsolete.

John Rooney11 min read
How to build your first Scrapy extension
Developer interest

How to build your first Scrapy extension

My Scrapy project now plays a triumphant fanfare when a crawl finishes clean and a sad trombone when it doesn't, which is also how I finally learned what Scrapy's extension points are actually for.

Ayan Pahwa11 min read
Now what exactly is loop engineering? And where do Anthropic's Fable 5 model and web scraping fit in?
Developer interest

Now what exactly is loop engineering? And where do Anthropic's Fable 5 model and web scraping fit in?

A viral clip from Claude Code's creator put a name to something a lot of us have been circling: loop engineering. Here's why web scraping may be its best-fit domain — and what that means in practice.

Ayan Pahwa12 min read
Automate deployment of your web scraper on VPS with Ubuntu 24.04 cloud-init
Developer interest

Automate deployment of your web scraper on VPS with Ubuntu 24.04 cloud-init

Your VPS is ready, but now you need to work through the same sequence you have run a dozen times before: apt update, apt install python3-pip, pip install scrapy, playwright install chromium, the Chromium dependency list that never installs cleanly on the first try, Redis, possibly Postgres, whatever else this particular project needs.

Ayan Pahwa9 min read
NotAnInterview: “I Have Superpowers Now"
Developer interest

NotAnInterview: “I Have Superpowers Now"

The problem was a project with 12,000 websites to crawl, and there’s no world where you write custom spiders for 12,000 websites, not with a human team and certainly not sustainably. So Javier built a workflow: a set of AI prompts that could analyze a website, figure out its structure, and generate a crawl configuration that a generic spider could then use.

Neha Setia Nagpal9 min read
Flatcar Linux for web scrapers: deploy immutable containers with just one config file
Developer interest

Flatcar Linux for web scrapers: deploy immutable containers with just one config file

The next time you spin up a VPS to give it a persistent home, you spend the better part of an afternoon rebuilding from memory. Here's a tool to help using Flatcar Linux

Ayan Pahwa11 min read

The Community · Newsletter

The best of Zyte and the data web, in your inbox.

One curated edition — new articles, product updates, and the stories shaping the data web. No noise.