Senior Anti-Bot / Scraping Engineer
A
AdthenaSearch intelligence
EU Remote (Please note, we can only accept applicants who are based in line with our remote hiring strategy: Spain, Italy, Portugal, Romania, Hungary, Czech Republic, Poland, Serbia, Lithuania, Bulgaria, Estonia, Latvia, Cyprus and Greece)Full-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSeleniumPlaywright
Requirements
- Have strong professional experience with Python or another main language, such as Go, Rust, or JavaScript.
- Have experience building production-grade scraping systems.
- Have hands-on experience scraping Google or similarly protected platforms.
- Understand HTTP, TLS, cookies, headers, redirects, and browser networking behaviour.
- Have experience with browser automation tools such as Playwright, Selenium, or Puppeteer, or equivalent frameworks.
- Know HTML parsing, DOM traversal, and data extraction techniques.
- Have experience handling rate limiting, CAPTCHA, IP rotation, and bot detection systems.
- Be comfortable with asynchronous and concurrent scraping architectures.
- Have experience running scrapers at scale in cloud environments.
- Have strong debugging skills and the ability to reason about complex failure modes.
- Have experience with headless browser fingerprinting and anti-bot evasion techniques.
- Containerisation and orchestration experience, such as Docker and Kubernetes, is a nice-to-have.
- Experience with distributed task systems and job queues is a nice-to-have.
- Background in data quality monitoring, anomaly detection, or search, advertising, or competitive intelligence data is a nice-to-have.
Responsibilities
- Design and maintain large-scale scraping systems that operate reliably against Google and other highly defended websites.
- Build scrapers for dynamic, JavaScript-heavy pages using browser automation and hybrid approaches.
- Adapt scraping systems to changes in page structure, request flows, and anti-automation mechanisms.
- Engineer extraction pipelines that prioritise data accuracy, consistency, and observability.
- Implement proxy, fingerprinting, and request-routing strategies to maximise stability and throughput.
- Monitor scrape health, detect failures, and debug production issues.
- Optimise performance, cost, and latency across scraping infrastructure.
- Work with data engineers, data scientists, and product teams to ensure scraped data is usable and trusted.
- Document scraping logic and operational runbooks.
View Full Description & ApplyYou'll be redirected to the employer's site
