Data Engineer - Web Scraping
C
CXGData engineering
India. Pakistan. Morocco. Taiwan. VietnamContractJunior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Fluent in English (French is a plus).
- Experience
- Minimum 2 years of experience in a similar role.
- Required Skills
- PythonSQLETLGitAirflowAzureNosqlData modelingdbt
Requirements
- Have a minimum of 2 years of experience in a similar role.
- Bring a strong academic background in a relevant field.
- Have data-engineering experience designing and implementing scalable data architectures.
- Have strong experience with ETL processes, data modeling, and data warehousing.
- Have expertise in relational SQL and NoSQL database technologies.
- Have knowledge of cloud platforms, particularly Azure.
- Understand data security measures and compliance standards.
- Have excellent Python experience for data engineering and automation.
- Have hands-on experience with Airflow and/or DBT; Airflow and DBT are preferred for data warehousing.
- Have experience with version control systems such as Git.
- Have experience with Terraform for infrastructure management.
- Be fluent in English; French is a plus.
Responsibilities
- Maintain and manage website-scraping configurations using Python.
- Monitor scraping configurations for errors and potential crashes.
- Oversee retrieved data to detect potential issues and blockages.
- Coordinate with stakeholders to understand scraping task requirements and report issues.
- Prepare and share periodic reports on scraping activities with stakeholders.
- Develop pipelines to ingest data into the Datalake and perform required transformations.
View Full Description & ApplyYou'll be redirected to the employer's site
