Data Engineer (Remote) - UK

View full description

Requirements:

• 3+ years of hands-on data engineering development experience, with deep expertise in Python, SQL, and working with SQL/NoSQL databases.
• Skilled in designing, building, and maintaining data pipelines, data warehouses, and leveraging AWS data services.
• Strong proficiency in DataOps methodologies and tools, including experience with CI/CD pipelines, containerized applications, and workflow orchestration using Apache Airflow.

Responsibilities:

• Design, develop, deploy and support data infrastructure, pipelines and architectures, contributing to an architectural vision that will scale up to be the world's leading research platform.
• Write clean, efficient, and maintainable code that powers data pipelines, workflows, and data operations in a production environment. Implement reliable, scalable, and well-tested solutions to automate data ingestion, transformation, and orchestration across systems.
• Manage and optimise key data infrastructure components within AWS, including Amazon Redshift, Apache Airflow for workflow orchestration and other analytical tools. You will be responsible for ensuring the performance, reliability, and scalability of these systems to meet the growing demands of data pipelines and analytics workloads.
• Overseeing configuration, monitoring, troubleshooting, and continuous improvement of our infrastructure to support delivering high-quality insights and analytics.