Data Engineer

Posted 11 days agoViewed

View full description

💎 Seniority level: Middle

📍 Location: EU

🔍 Industry: Decentralized Finance

🏢 Company: P2P. org

🗣️ Languages: English

🪄 Skills: PythonSQLGCPGitAirflowClickhouseData engineeringData modeling

Requirements:

Strong knowledge: Python, SQL- any syntax, preferably BQ, Clickhouse, Postgres
Production experience with Airflow, Clickhouse or BigQuery, DBT, git
General understanding and experience with GCP or AWS
English level: B2+

Responsibilities:

Perform technical and business tasks from analysts related to our core tools
Participate in code reviews of analysts and identifying suboptimal processes
Monitor load and alerts from our services
Take care about Data Platform
Write DBT models for Core Datamarts

Apply

Related Jobs

Apply

🔥 Big Data Engineer

Posted 1 day ago

📍 Spain

🔍 Software Development

🏢 Company: Plain Concepts👥 251-500 Consulting Apps Mobile Apps Information Technology Mobile

🔧 Requirements

3 years of experience in data engineering.
Strong experience with Python or Scala and Spark, processing large datasets.
Solid experience in Cloud platforms (Azure or AWS).
Hands-on experience building data pipelines (CI/CD).
Experience with testing (unit, integration, etc.).
Knowledge of SQL and NoSQL databases.

💡 Responsibilities

Participating in the design and development of Data solutions for challenging projects.
Develop projects from scratch with minimal supervision and strong team collaboration.
Be a key player in fostering best practices, clean, and reusable code.
Develop ETLs using Spark (Python/Scala).
Work on cloud-based projects (Azure/AWS).
Build scalable pipelines using a variety of technologies.

AWSPythonSQLAgileCloud ComputingETLAzureData engineeringNosqlSparkCI/CDScala

Posted 1 day ago

Apply

🔥 Fullstack Data Engineer - Woman/Man/Non Binary

Posted 1 day ago

📍 France

🧭 Full-Time

🔍 Data Engineering

🏢 Company: Filigran

🔧 Requirements

You’ve already trained and deployed machine learning models in production environments
Proficient in Python, and familiar with its full-stack usage (packaging, logging, FastAPI, Flask)
Strong autonomy: you can take a scoped project and drive it independently
Solid understanding of software engineering best practices: versioning, code reviews, pair programming
Product-oriented mindset: you care about UX, data value, and user outcomes

💡 Responsibilities

Design and prototype ML models and AI features (e.g. Named Entity Recognition for CTI enrichment)
Translate concepts into PoCs and MVPs to validate value
Maintain and improve our growing AI/ML pipelines
Collaborate closely with engineers to ensure smooth downstream integration into our open-source platforms
Conduct technical watch and apply state-of-the-art practices
Contribute to the growth of our enterprise data platform
Ensure code quality, reliability and test automation via reviews and collaboration

DockerPythonSQLApache AirflowFlaskGitKubernetesMachine LearningMLFlowNumpyData engineeringFastAPIREST APIPandasCI/CDData visualizationData modelingSoftware Engineering

Posted 1 day ago

Apply

🔥 Senior Data Engineer

Posted 2 days ago

📍 Germany, Spain, Portugal, Greece

🏢 Company: WorkMotion👥 101-250💰 $10,000,000 Series B almost 3 years agoCompliance Human Resources Employee Benefits

🔧 Requirements

3-5 years of professional experience in Data Engineering or Software Development with a focus on data
Strong knowledge of Python and SQL; and PySpark
Hands-on experience with AWS services (Glue, S3, Athena, EC2)
Experience with Apache Airflow, preferably in a Dockerized/cloud-native environment
Familiarity with Delta Lake or similar data lake frameworks
Proficiency with source control (GitHub) and CI/CD workflows
Strong understanding of data modeling, ETL best practices, and data pipeline performance optimization

💡 Responsibilities

Design, build, and maintain scalable ETL pipelines using Apache Airflow and AWS Glue (Spark)
Work with a range of data sources including Salesforce, NetSuite, PostgreSQL, and MongoDB
Develop and optimize PySpark jobs for large-scale data transformation and analytics
Manage data lake infrastructure using Delta Lake on S3 with Athena as the query layer
Ensure data quality, performance, and reliability through monitoring, testing, and documentation
Collaborate with analytics, product, and engineering teams to define data requirements
Contribute to CI/CD workflows with GitHub and deployment automation
Participate in architectural discussions and advocate for best practices in data engineering

AWSDockerPythonSoftware DevelopmentSQLApache AirflowETLGitData engineeringSparkCI/CDData modeling

Posted 2 days ago

Apply

🔥 Data Engineer: Data Infrastructure (Remote)

Posted 5 days ago

📍 Portugal

💸 90000.0 - 110000.0 USD per year

🔍 Software Development

🏢 Company: Constructor

🔧 Requirements

Solid skills in any programming language (ideally Python)
Big data engineering
Web services
Cloud providers (ideally AWS)

💡 Responsibilities

Implement a data model for our tables (bronze, silver, gold) to incapsulate parsing logic, lower the costs and provide better abstractions
Power our recommendation system with a fast, global data store.
Improve the build system in our repositories to speed up test execution, job deployment and thus improving the developers productivity
Optimize Spark pipelines, ClickHouse tables/queries, and streaming processing in AWS Lambda.

AWSPythonSQLCloud ComputingETLGitKubernetesClickhouseData engineeringSparkCI/CDData visualizationData modelingData management

Posted 5 days ago

Apply

🔥 Senior Data Engineer - Data Platform 🇪🇺 (Remote)

Posted 5 days ago

📍 Europe

🧭 Full-Time

🔍 Fintech

🔧 Requirements

NOT STATED

💡 Responsibilities

Design our data platform, from data ingestion and validation to transport, storage, and exposure to consumers.
Collaborate with other data engineering teams focused on delivering analytics services, sharing technical knowledge and defining best practices, and guiding them in the configuration of core systems such as data warehousing and orchestration.
Work hand in hand with machine learning engineers to design and scale automation and intelligence features into the Pennylane application.
Partner with software engineers to ensure data quality and integrity, acting as a role model to establish data as a core part of our production systems.
Drive optimizations on our core components to maintain an effective balance between scalability and costs.
Shape an engineering culture with high standards in the data team by defining best practices, architecture patterns and security measures.

AWSPythonSQLApache AirflowETLData engineeringData visualizationData modelingData analyticsData management

Posted 5 days ago

Apply

🔥 Senior Data Engineer - Data Platform (Remote)

Posted 6 days ago

📍 Europe

🔍 Fintech

🔧 Requirements

Experience with AWS services
Experience with infrastructure as code, batch processing, scheduling, data warehousing
Understanding of data engineering best practices

💡 Responsibilities

Design our data platform, from data ingestion and validation to transport, storage, and exposure to consumers.
Collaborate with other data engineering teams focused on delivering analytics services, sharing technical knowledge and defining best practices, and guiding them in the configuration of core systems such as data warehousing and orchestration.
Work hand in hand with machine learning engineers to design and scale automation and intelligence features into the Pennylane application.
Partner with software engineers to ensure data quality and integrity, acting as a role model to establish data as a core part of our production systems.
Drive optimizations on our core components to maintain an effective balance between scalability and costs.
Shape an engineering culture with high standards in the data team by defining best practices, architecture patterns and security measures.

AWSPostgreSQLPythonSQLApache AirflowData AnalysisETLKafkaSnowflakeAlgorithmsData engineeringData StructuresSparkCI/CDRESTful APIsMicroservicesData visualizationData modelingScriptingData analyticsData management

Posted 6 days ago

Apply

🔥 Senior Data Engineer - Streaming Platform

Posted 7 days ago

📍 EMEA countries

🧭 Full-Time

🔍 Mobile Games and Apps

🏢 Company: Voodoo

🔧 Requirements

Extensive experience in data or backend engineering, with at least 2+ years building real-time data pipelines.
Proficiency with stream processing frameworks like Flink, Spark Structured Streaming, Beam, or similar.
Strong programming experience in Java, Scala, or Python, with a focus on distributed systems.
Deep understanding of event streaming and messaging platforms such as GCP Pub/Sub, AWS Kinesis, Apache Pulsar, or Kafka — including performance tuning, delivery guarantees, and schema management.
Solid experience operating data services in Kubernetes, including Helm, resource tuning, and service discovery.
Experience with Protobuf/Avro, and best practices around schema evolution in streaming environments.
Familiarity with CI/CD workflows and infrastructure-as-code (e.g., Terraform, ArgoCD, CircleCI).
Strong debugging skills and a bias for building reliable, self-healing systems.

💡 Responsibilities

Design, implement, and optimize real-time data pipelines handling billions of events per day with strict SLAs.
Architect data flows for bidstream data, auction logs, impression tracking and user behavior data.
Build scalable and reliable event ingestion and processing systems using Kafka, Flink, Spark Structured Streaming, or similar technologies.
Operate data infrastructure on Kubernetes, managing deployments, autoscaling, resource limits, and high availability.
Collaborate with backend to integrate OpenRTB signals into our data platform in near real-time.
Ensure high-throughput, low-latency processing, and system resilience in our streaming infrastructure.
Design and manage event schemas (Avro, Protobuf), schema evolution strategies, and metadata tracking.
Implement observability, alerting, and performance monitoring for critical data services.
Contribute to decisions on data modeling and data retention strategies for real-time use cases.
Mentor other engineers and advocate for best practices in streaming architecture, reliability, and performance.
Continuously evaluate new tools, trends, and techniques to evolve our modern streaming stack.

Backend DevelopmentPythonSQLGCPJavaKafkaKubernetesAlgorithmsData engineeringData StructuresSparkCI/CDRESTful APIsLinuxTerraformMicroservicesScalaData modelingDebugging

Posted 7 days ago

Apply

🔥 Lead Data Engineer(Snowflake/DBT)

Posted 9 days ago

📍 UK, India, Germany

🧭 Full-Time

🔍 Fintech

🏢 Company: Careers at Tide

🔧 Requirements

4+ years of extensive development experience using snowflake or similar data warehouse technology
Working experience with dbt and other technologies of the modern data stack, such as Snowflake, Apache Airflow, Fivetran, AWS, git ,Looker
Experience in agile processes, such as SCRUM
Extensive experience in writing advanced SQL statements and performance tuning them
Experience in Data Ingestion techniques using custom or SAAS tool like fivetran
Experience in data modelling and can optimise existing/new data models
Experience in data mining, data warehouse solutions, and ETL, and using databases in a business environment with large-scale, complex datasets
Experience architecting analytical databases (in Data Mesh architecture) is added advantage
Experience working in agile cross-functional delivery team
High development standards, especially for code quality, code reviews, unit testing, continuous integration and deployment
Strong technical documentation skills and the ability to be clear and precise with business users
Business-level of English and good communication skills
Basic understanding of various systems across the AWS platform ( Good to have )
Preferably, you have worked in a digitally native company, ideally fintech

💡 Responsibilities

Developing end to end ETL/ELT Pipeline working with Data Analysts of business Function.
Designing, developing, and implementing scalable, automated processes for data extraction, processing, and analysis in a Data Mesh architecture
Mentoring Fother Junior Engineers in the Team
Be a “go-to” expert for data technologies and solutions
Ability to provide on the ground troubleshooting and diagnosis to architecture and design challenges
Troubleshooting and resolving technical issues as they arise
Looking for ways of improving both what and how data pipelines are delivered by the department
Translating business requirements into technical requirements, such as entities that need to be modelled, DBT models that need to be build, timings, tests and reports
Owning the delivery of data models and reports end to end
Perform exploratory data analysis in order to identify data quality issues early in the process and implement tests to ensure prevent them in the future
Working with Data Analysts to ensure that all data feeds are optimised and available at the required times. This can include Change Capture, Change Data Control and other “delta loading” approaches
Discovering, transforming, testing, deploying and documenting data sources
Applying, help defining, and championing data warehouse governance: data quality, testing, coding best practices, and peer review
Building Looker Dashboard for use cases if required

AWSPythonSQLAgileApache AirflowBusiness IntelligenceData AnalysisData MiningETLGitSCRUMSnowflakeData engineeringTroubleshootingData visualizationData modeling

Posted 9 days ago

Apply

🔥 Senior Data Engineer

Posted 9 days ago

📍 Worldwide

🧭 Full-Time

💸 140000.0 - 175000.0 USD per year

🔍 Software Development

🏢 Company: Figment👥 11-50 Hospitality Travel Accommodations Art

🔧 Requirements

Extensive experience with data engineering, including building and managing data pipelines and ETL processes.
Proficiency in the Python programming language and SQL.
Experience developing highly concurrent and performant applications ensuring scalability and efficient resource utilization in distributed or multi-threaded systems.
Experience implementing robust microservices following best practices in error handling, logging, and testing for production-grade systems.
Experience with using CI/CD pipelines for automated data infrastructure provisioning and application deployment.
Experience with the data orchestration tool Dagster or Airflow.
Experience designing and orchestrating complex DAGs to manage dependencies, triggers, and retries for data workflows, ensuring reliable and efficient pipeline execution.
Experience with the data transformation tool DBT.
Experience designing and implementing complex data transformations using advanced DBT models, materializations, and configurations to streamline data workflows and improve performance.
Experience optimizing and troubleshoot DBT pipelines for scale, ensuring that transformations run efficiently in production environments, handling large datasets without issues.
Experience with cloud data warehousing platforms (e.g. Snowflake)
Experience architecting and optimizing Snowflake environments for performance, including designing partitioning strategies, clustering keys, and storage optimizations for cost-effective scaling.
Has an understanding of security and governance policies within Snowflake, including data encryption, access control, and audit logging to meet compliance and security best practices.

💡 Responsibilities

Implement and maintain reliable data pipelines and data storage solutions.
Implement data modeling and integrate technologies according to project needs.
Manage specific data pipelines and oversees the technical aspects of data operations
Ensure data processes are optimized and align with business requirements
Identify areas for process improvements and suggests tools and technologies to enhance efficiency
Continuously improve data infrastructure automation, ensuring reliable and efficient data processing.
Develop and maintain data pipelines and ETL processes using technologies such as Dagster and DBT to ensure efficient data flow and processing.
Automate data ingestion, transformation, and loading processes to support blockchain data analytics and reporting.
Utilize Snowflake data warehousing solutions to manage and optimize data storage and retrieval.
Collaborate with Engineering Leadership and Product teams to articulate data strategies and progress.
Promote best practices in data engineering, cloud infrastructure, networking, and security.

PythonSQLCloud ComputingETLSnowflakeData engineeringCI/CDRESTful APIsMicroservicesData modeling

Posted 9 days ago

Apply

🔥 Data Engineer

Posted 9 days ago

📍 England, Scotland, Portugal, Poland, Spain

🔍 Warehouse Automation

🏢 Company: Locus Robotics👥 251-500💰 $117,000,000 Series F over 2 years agoWarehousing Logistics Industrial Automation E-Commerce Warehouse Automation Robotics

🔧 Requirements

3+ years of experience developing and deploying production-grade Python software.
3+ years of experience with Python and high-performance data libraries such as Polars and Pandas.
Proficiency with JavaScript, SQL, and KQL.
Experience with Extract, Transform, Load (ETL), Data Streaming, and Reconciliation.
Experience building and maintaining deployment pipelines, including DevOps tools like Ansible, and containerization with Docker.
Proficiency with cloud platforms (AWS or Azure) for deploying and scaling data systems.
Highly desired experience with Azure, particularly Lakehouse and Eventhouse architectures.
Experience with relevant infrastructure and tools including NATS, Power BI, Apache Spark/Databricks, and PySpark.
Hands-on experience with data warehousing methodologies and optimization libraries (e.g., OR-Tools).
Experience with log analysis, forensic debugging, and system performance tuning.
Exposure to cloud-based systems.
Familiarity with Agile/SCRUM methodologies in collaborative development workflows.

💡 Responsibilities

Develop and maintain Python-based systems deployed across remote platforms.
Contribute to and improve data pipelines, ensuring reliable and efficient system updates.
Build and enhance features for real-time data analysis and system monitoring to ensure high uptime and efficiency.
Collaborate with data scientists and engineers to support advanced analytics and machine learning workflows.
Support the migration of our codebase toward machine learning capabilities by building scalable, maintainable solutions.
Analyze system logs and performance to debug issues and optimize operations using forensic analysis tools.

AWSDockerPythonSQLAgileCloud ComputingData AnalysisETLJavascriptMachine LearningAzureData engineeringREST APIPandasCI/CDDevOpsAnsibleDebugging

Posted 9 days ago

Apply