Data Engineer

Posted 3 months agoViewed

💎 Seniority level: Middle, At least 3 years

📍 Location: United Kingdom

🔍 Industry: Fintech

🏢 Company: Kody👥 51-100💰 $20,000,000 Series A 5 months agoMobile Payments E-Commerce Point of Sale Retail Technology FinTech

🗣️ Languages: English

⏳ Experience: At least 3 years

🪄 Skills: AWSPostgreSQLPythonSQLBashData AnalysisETLGCPGitKafkaBusiness OperationsData engineeringPostgresCollaboration

Requirements:

At least 3 years of experience in data engineering or a related field.
Strong SQL skills and experience with relational databases (PostgreSQL preferred).
Experience with ETL tools and processes.
Working knowledge of data modelling and database design principles.
Experience with scripting languages (Python, Bash).
Experience with cloud-based data platforms (AWS, GCP, we use both).
Proficient in data warehousing concepts and methodologies.
Experience with data visualisation tools (Tableau, Power BI, Looker; we use Looker and it would be fantastic if you do as well).
Experience with version control systems (Git).
Strong analytical and problem-solving skills.
Excellent communication and collaboration skills.

Responsibilities:

Design, develop, and maintain data architectures and infrastructure.
Collaborate with cross-functional teams to understand data needs and develop solutions.
Implement ETL processes to extract, transform, and load data from various sources.
Create and optimize SQL queries for data analysis and reporting.
Implement and maintain data pipelines and workflows.
Perform data validation and quality assurance tasks.
Monitor and optimize database performance.
Stay up-to-date with emerging trends and technologies in the data engineering field.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer

Posted about 3 hours ago

📍 Europe and eastern US

🔍 Crypto data analytics

Proven experience with building robust data pipelines and using orchestration tools, for example with DBT and Prefect.
Proficiency with data warehouses, with expertise in at least one of Trino, Snowflake or Clickhouse.
Strong ability to independently analyze, debug, and fix data pipeline issues.
Deployment and infrastructure experience in public cloud platforms (e.g., Kubernetes).
Solid foundation in computer science fundamentals and system design.
Ability to work collaboratively in a remote setting, contributing to a positive and inclusive team culture.

Build data pipelines that power some of the most popular data on Dune.
Support analysts to build new powerful datasets.
Assume ownership of data transformation pipelines that take community contributed SQL transformations as input.
Orchestrate robust pipelines that are resilient to user error in a fast-paced contribution environment.
Understand and prioritize the needs of our diverse community from amateur data sleuths to professional data scientists and engineers.

PythonSQLETLKubernetesSnowflakeClickhouseData engineeringDebugging

Posted about 3 hours ago

Apply

🔥 Senior Staff / Principal Data Engineer

Posted 5 days ago

📍 Copenhagen, London, Stockholm, Berlin, Madrid, Montreal, Lisbon, 35 other countries

🧭 Full-Time

🔍 Financial Technology

Strong background in building and managing data infrastructure at scale.
Expertise in Python, AWS, dbt, Airflow, and Kubernetes.
Ability to translate business and product requirements into technical data solutions.
Experience in mentoring and fostering collaboration within teams.
Curiosity and enthusiasm for experimenting with new technologies to solve complex problems.
Hands-on experience with modern data tools and contributing to strategic decision-making.

Partnering with product and business teams to develop data strategies that enable new features and improve user experience.
Driving key strategic projects across the organisation, dipping in and out as needed to provide leadership and hands-on support.
Supporting multiple teams across Pleo in delivering impactful data and analytics solutions.
Building data products that directly support Pleo's product roadmap and business goals.
Collaborating with the VP of Data and other data leaders to set the vision for Pleo’s data strategy and ensure alignment with company objectives.
Enhancing our data infrastructure and pipelines to improve scalability, performance, and data quality.
Experimenting with and implementing innovative technologies to keep Pleo’s data stack at the forefront of the industry.
Mentoring engineers, analysts, and data scientists to foster growth and build a world-class data team.

AWSPythonApache AirflowKubernetesData engineering

Posted 5 days ago

Apply

🔥 Senior Data Engineer - Compute and Storage Infrastructure

Posted 7 days ago

📍 US, Europe

🧭 Full-Time

💸 175000.0 - 205000.0 USD per year

🔍 Cloud computing and AI services

🏢 Company: CoreWeave💰 $642,000,000 Secondary Market about 1 year agoCloud Computing Machine Learning Information Technology Cloud Infrastructure

5+ years of experience with Kubernetes and Helm, with a deep understanding of container orchestration.
Hands-on experience administering and optimizing clustered computing technologies on Kubernetes, such as Spark, Trino, Flink, Ray, Kafka, StarRocks or similar.
5+ years of programming experience in C++, C#, Java, or Python.
3+ years of experience scripting in Python or Bash for automation and tooling.
Strong understanding of data storage technologies, distributed computing, and big data processing pipelines.
Proficiency in data security best practices and managing access in complex systems.

Architect, deploy, and scale data storage and processing infrastructure to support analytics and data science workloads.
Manage and maintain data lake and clustered computing services, ensuring reliability, security, and scalability.
Build and optimize frameworks and tools to simplify the usage of big data technologies.
Collaborate with cross-functional teams to align data infrastructure with business goals and requirements.
Ensure data governance and security best practices across all platforms.
Monitor, troubleshoot, and optimize system performance and resource utilization.

PythonBashKubernetesApache Kafka

Posted 7 days ago

Apply

🔥 Senior Staff / Principal Data Engineer

Posted 8 days ago

📍 Copenhagen, London, Stockholm, Berlin, Madrid, Montreal, Lisbon

🧭 Full-Time

🔍 Financial solutions

🏢 Company: Pleo👥 501-1000💰 $42,922,001 Debt Financing 9 months ago🫂 Last layoff about 2 years agoMobile Payments Financial Services Payments Information Technology FinTech

Strong background in building and managing data infrastructure at scale.
Expertise in Python, AWS, dbt, Airflow, and Kubernetes.
Hands-on experience with modern data tools and technology.
Ability to translate business and product requirements into technical data solutions.
Strong leadership and mentoring capabilities.

Partner with product and business teams to develop data strategies that enable new features and improve user experience.
Drive key strategic projects across the organisation, providing leadership and hands-on support as needed.
Support multiple teams in delivering impactful data and analytics solutions.
Build data products that align with Pleo's product roadmap and business goals.
Collaborate with the VP of Data and data leaders to set a vision for Pleo’s data strategy.
Enhance data infrastructure and pipelines for scalability, performance, and data quality.
Experiment with and implement innovative technologies to maintain a cutting-edge data stack.
Mentor engineers, analysts, and data scientists to foster a world-class data team.

AWSPythonApache AirflowETLKubernetesData engineeringData management

Posted 8 days ago

Apply

🔥 Senior Data Engineer

Posted 9 days ago

📍 South Africa, Mauritius, Kenya, Nigeria

🔍 Technology, Marketplaces

BSc degree in Computer Science, Information Systems, Engineering, or related technical field or equivalent work experience.
3+ years related work experience.
Minimum of 2 years experience building and optimizing ‘big data’ data pipelines, architectures and maintaining data sets.
Experienced in Python.
Experienced in SQL (PostgreSQL, MS SQL).
Experienced in using cloud services: AWS, Azure or GCP.
Proficiency in version control, CI/CD and GitHub.
Understanding/experience in Glue and PySpark highly desirable.
Experience in managing data life cycle.
Proficiency in manipulating, processing and architecting large disconnected data sets for analytical requirements.
Ability to maintain and optimise processes supporting data transformation, data structures, metadata, dependency and workload management.
Good understanding of data management principles - data quality assurance and governance.
Strong analytical skills related to working with unstructured datasets.
Understanding of message queuing, stream processing, and highly scalable ‘big data’ datastores.
Strong attention to detail.
Good communication and interpersonal skills.

Suggest efficiencies and execute on implementation of internal process improvements in automating manual processes.
Implement enhancements and new features across data systems.
Improve streamline processes within data systems with support from Senior Data Engineer.
Test CI/CD process for optimal data pipelines.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Highly efficient in ETL processes.
Develop and conduct unit tests on data pipelines as well as ensuring data consistency.
Develop and maintain automated monitoring solutions.
Support reporting and analytics infrastructure.
Maintain data quality and data governance as well as upkeep of overall maintenance of data infrastructure systems.
Maintain data warehouse and data lake metadata, data catalogue, and user documentation for internal business users.
Ensure best practice is implemented and maintained on database.

AWSPostgreSQLPythonSQLETLGitCI/CD

Posted 9 days ago

Apply

🔥 [CMI] Senior Cloud Data Engineer

Posted 29 days ago

📍 UK

🧭 Full-Time

🔍 Technology solutions

5+ years of commercial experience in Data Engineering.
Experience in a senior technical leadership role.
Strong expertise in modern data warehouse architectures, with experience in Snowflake preferred.
Advanced knowledge of Python for data processing and automation.
Expert-level SQL skills for complex data transformations and optimization.
Extensive experience in designing and implementing ETL/ELT pipelines.
Deep expertise in Azure cloud services and data platform components.
Strong background in Infrastructure as Code - Terraform.
Proven experience with GIT and CI/CD tools (Azure DevOps).
Track record of successfully implementing enterprise-scale data solutions.
Expertise in data visualization tools, particularly Power BI.

Lead and shape the newly created Data Engineers team.
Architect and implement enterprise-scale data solutions.
Drive technical excellence and establish best practices for ETL/ELT processes.
Mentor team members and provide technical leadership across multiple workstreams.
Collaborate with stakeholders to translate business requirements into scalable data solutions.

PythonSQLETLGitSnowflakeAzureData engineeringCI/CDTerraform

Posted 29 days ago

Apply

🔥 Data Engineer - Technical Intelligence

Posted about 1 month ago

📍 United Kingdom

🔍 Blockchain

🏢 Company: IO Global

BSc/MSc in a Computer Science field or equivalent practical experience.
4+ years of work experience in data and software engineering.
Knowledge of big data processing platforms like Spark and Google BigQuery.
Experience with cloud services management, specifically AWS or GCP.
Strong knowledge of relational databases and optimized SQL.
Familiarity with container orchestration tools, CI/CD pipelines, and infrastructure as code.
Ability to communicate well with both technical and non-technical partners.

You will be part of the Technical Intelligence (TechInt) team, responsible for maintaining and setting up data solutions and services.
Aid in the maturation of the data ingestion pipeline and processes.
Create a state-of-the-art cloud-native data warehouse.
Develop and maintain automated data ingestion pipelines from diverse sources.
Research existing datasets for relevance and simplify existing data pipelines.
Collaborate with data scientists to operationalize ML models.
Work closely with leadership to align with strategy and roadmap.

AWSDockerPostgreSQLSQLApache AirflowBlockchainCloud ComputingDynamoDBGCPKubernetesMachine LearningData engineeringSparkCI/CDTerraformData modeling

Posted about 1 month ago

Apply

🔥 Senior/Staff Data Engineer

Posted about 1 month ago

📍 UK

🔍 Advertising

Ability to take an ambiguously defined task, and break it down into actionable steps.
Ability to follow through complex projects to completion, both by independent implementation and by coordinating others.
Deep understanding of algorithm and software design, concurrency, and data structures.
Experience in implementing probabilistic or machine learning algorithms.
Experience in designing scalable distributed systems.
High GPA from a well-respected Computer Science program or equivalent experience.

Design modular and scalable real-time data pipelines to handle huge datasets.
Suggest, implement, and coordinate architectural improvements for big data ML pipelines.
Understand and implement custom ML algorithms in a low latency environment.
Work on microservice architectures that run training, inference, and monitoring on thousands of ML models concurrently.

AWSPythonApache AirflowMachine LearningAlgorithmsData engineeringData Structures

Posted about 1 month ago

Apply

🔥 Senior Data Engineer

Posted about 1 month ago

📍 United States, United Kingdom, Spain, Estonia

🔍 Identity verification

🏢 Company: Veriff👥 501-1000💰 $100,000,000 Series C almost 3 years ago🫂 Last layoff over 1 year agoArtificial Intelligence (AI)Fraud Detection Information Technology Cyber Security Identity Management

Expert-level knowledge of SQL, particularly with Redshift.
Strong experience in data modeling with an understanding of dimensional data modeling best practices.
Proficiency in data transformation frameworks like dbt.
Solid programming skills in languages used in data engineering, such as Python or R.
Familiarity with orchestration frameworks like Apache Airflow or Luigi.
Experience with data from diverse sources including RDBMS and APIs.

Collaborate with business stakeholders to design, document, and implement robust data models.
Build and optimize data pipelines to transform raw data into actionable insights.
Fine-tune query performance and ensure efficient use of data warehouse infrastructure.
Ensure data reliability and quality through rigorous testing and monitoring.
Assist in migrating from batch processing to real-time streaming systems.
Expand support for various use cases including business intelligence and analytics.

PythonSQLApache AirflowETLData engineeringJSONData modeling

Posted about 1 month ago

Apply

🔥 Senior Data Engineer (SQL, Fabric, ADF)

Posted about 1 month ago

📍 Ireland, United Kingdom

🔍 IT, Digital Transformation

🏢 Company: Tekenable👥 51-100 Information Technology Enterprise Software Software

Experience with the Azure Intelligent Data Platform, including Data Lakes, Data Factory, Azure Synapse, Azure SQL, and Power BI.
Knowledge of Microsoft Fabric.
Proficiency in SQL and Python.
Understanding of data integration and ETL processes.
Ability to work with large datasets and optimize data systems for performance and scalability.
Experience working with JSON, CSV, XML, Open API, RESTful API integration and OData v4.0.
Strong knowledge of SQL and experience with relational databases.
Experience with big data technologies like Hadoop, Spark, or Kafka.
Familiarity with cloud platforms such as Azure.
Bachelor's degree in Computer Science, Engineering, or a related field.

Design, develop, and maintain scalable data pipelines.
Collaborate with data analysts to understand their requirements.
Implement data integration solutions to meet business needs.
Ensure data quality and integrity through testing and validation.
Optimize data systems for performance and scalability.

PythonSQLETLHadoopKafkaAzureSparkJSON

Posted about 1 month ago

Apply

Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities