Developer to Developer consultation for top challenges

Clients interested in migrating to Zyte API can benefit from advisory services to ensure a smooth migration.

• Basic orientation in Zyte API & Scrapy Cloud - highlighting key features and how to get started
• Zyte API & Scrapy Cloud usage examples - how to use our APIs effectively
• Troubleshooting processes - some basic tips on how to approach troubleshooting; e.g. checking if any anti-ban has been introduced or changed

Learn coding best practices for web scraping. Training in Python code style best practices - using tools like Black, Isort, Pre-commit, following PEP8 style, and using linters to ensure code adheres to industry standards and best practices.

• Setup standardized tools -setup Black, Isort, pre-commit, Scrapy Cloud and Spidermon
• Introduction to Spidermon - overview of Spidermon and basic monitoring
• Workshop on Spidermon -usage, error checking, email notification, sentry (or similar), Scrapy stats, reporting and advanced monitoring

A thorough analysis of your web scraping solution to identify potential points for optimization.

• Audit of current web scraping - walkthrough of code and a full review of Crawling Strategy and Infrastructure
• Report on audit findings to identify primary bottlenecks (e.g. velocity, parallel processes, etc.). Identify inconsistencies, risks, and cost-saving recommendations
• Workshop on audit findings to optimize your web scraping solution

Introduction to Data Quality Assurance and automated & manual data quality practices

• Schema Validation - turning the inherent structured nature of your data into fast, whole-dataset validation per data point
• Advanced data quality assurance with Python. Take data quality assurance further with advanced techniques to uncover issues and anomalies
• Visualizing Data Quality - Turn advanced data quality assurance into charts and visualisations to highlight trends and tell the data quality story

A thorough analysis of your current Data Quality Assurance setup to identify potential opportunities for improvement.

• Audit of current Data Quality setup. Walkthrough of code and tools accompanied by Zyte QA Engineer. Review of monitoring strategy AND response capabilities
• Report on audit findings to identify processes and tooling inefficiencies, risks, and Data  Quality improvement recommendations
• Workshop on audit findings to optimize your data quality solution