Senior Python Data Scraping Engineer
New
J
JobgetherData Engineering
Based in IndiaContractSenior
SalaryUp to $30 per hour equivalent
Apply NowOpens the employer's application page
Job Details
- Languages
- Upper-intermediate (B2) English proficiency or higher
- Experience
- Minimum of 5 years
- Required Skills
- AWSDockerPythonSeleniumRESTful APIsLangChain
Requirements
- Minimum of 5 years of professional experience in data engineering, web scraping, automation, software development, or a related technical field.
- Strong expertise in Python for web scraping using frameworks such as BeautifulSoup, Selenium, or similar technologies.
- Proven experience extracting data from dynamic websites, JavaScript-rendered applications, REST APIs, and complex HTML structures.
- Solid understanding of data cleaning, normalization, validation, and delivery of structured datasets in formats such as CSV, JSON, or Google Sheets.
- Experience overcoming anti-bot protections, managing proxies, and maintaining stable scraping solutions for changing website architectures.
- Hands-on experience with cloud platforms such as AWS (or equivalent), Docker, and scalable automation workflows.
- Familiarity with modern AI and LLM frameworks such as LangChain, OpenRouter, or similar technologies is highly desirable.
- Strong analytical thinking, troubleshooting skills, attention to detail, and the ability to work independently with minimal supervision.
- Bachelor's or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related technical discipline is preferred.
- Upper-intermediate (B2) English proficiency or higher is required.
Responsibilities
- Design, develop, and maintain end-to-end web scraping workflows that efficiently collect structured data from complex and dynamic websites.
- Build reliable data extraction solutions using Python and modern scraping frameworks while adapting techniques to handle JavaScript-rendered content, APIs, and evolving website structures.
- Implement scalable data collection processes through automation, batching, parallel processing, and monitoring to ensure efficient execution across large datasets.
- Validate, clean, normalize, and structure extracted data while maintaining strict quality standards and ensuring consistency across multiple data sources.
- Troubleshoot anti-bot mechanisms, optimize scraping performance, and continuously improve the reliability and resilience of extraction pipelines.
- Utilize cloud infrastructure, containerization technologies, and automation frameworks to support scalable and maintainable data processing workflows.
- Collaborate effectively within a distributed project environment, documenting processes and delivering high-quality datasets that meet project specifications.
View Full Description & ApplyYou'll be redirected to the employer's site
