Why is remote work so nice? Let's try to see!
Apply
Senior Data Engineer
Posted 6 months agoViewed
💎 Seniority level: Senior
💸 Salary: 5700.0 - 7400.0 EUR per month
🔍 Industry: Software Development
🏢 Company: Trafi👥 101-250💰 Series B over 4 years agoCar SharingRide SharingTransportationPublic TransportationAppsInformation TechnologyMobileTravelSoftware
Requirements:
- Experience in designing and operating data processing solutions, covering full lifecycle from collection to ingestion, storage, transformation and reporting
- Hands-on experience using public cloud infrastructure, preferably AWS
- Strong SQL experience. Ideally with modern data warehouses like BigQuery, Snowflake, Redshift, Firebolt
- Good command of at least one general purpose programming language, ideally Python
- Knowledge of Unix shell scripting, Docker, Git version control system.
Responsibilities:
- Design and maintain our data pipelines by employing engineering best practices documentation, testing, cost optimisation, version control
- Work in Data Platform team to transform business and analytical needs into accurate and reusable data models in our data warehouse
- Identify gaps in data collection, diagnose and fix data discrepancies and maintain model code to fulfil business requirements and keep consistent ETL logic
- Participate in transition towards Data Mesh architecture by building related tooling, infrastructure and providing consultations to domain teams
- Stay up to date with the latest technologies and trends to improve our existing data analytics and data platform stack
Related Jobs
Apply
🔥 Senior Data Engineer
Posted 1 day ago
🔍 Software Development
- Degree in a related field and 7+ years of data engineering experience.
- Proficiency in tools and languages, such as AWS, dbt, Snowflake, Git, R, Python, SQL, SQL Server, and Snowflake.
- Strong project management skills and the ability to communicate complex concepts effectively.
- Strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.
- Adept at conveying complex data insights through a robust understanding of data management systems, warehouse methodologies, data quality standards, data modeling techniques, governance protocols, and advanced analytics.
- Familiarity with agile work environments and scrum ceremonies.
- Strong business acumen and experience in aligning data initiatives with business objectives.
- Contribute to the strategic vision for data engineering and participate in the architectural design and development of new and complex data solutions, focusing on scalability, performance, and hands-on implementation.
- Design and implement new data systems and infrastructure to ensure the reliability and scalability of data systems by actively contributing to day-to-day engineering tasks.
- Influence key decisions regarding the data technology stack, infrastructure, and tools while actively engaging in hands-on engineering efforts in the creation and deployment of new data architectures and workflows.
- Set coding standards and best practices for the Data Engineering & Operations team, conducting and participating in code reviews to maintain high-quality, consistent code.
- Work closely with database developers, software development, product management, and AI/ML developers to align data initiatives with Assent’s organizational goals.
- Collaborate with team members to monitor progress, adjust priorities, and meet project deadlines and objectives.
- Identify opportunities for internal process improvements, including automating manual processes and optimizing data delivery.
- Proactively support peers in continuous learning by providing technical guidance and training on data engineering development, analysis, and execution.
- Be familiar with corporate security policies and follow the guidance set out by processes and procedures of Assent.
Posted 1 day ago
ApplyApply
🔥 Senior Data Engineer
Posted 2 days ago
📍 Canada
🔍 Software Development
🏢 Company: Annex It Solutions
- 5+ years of experience in data engineering or a related field.
- Strong expertise in data warehousing concepts and technologies, such as Snowflake, Redshift, or similar.
- Proficiency in programming languages such as Python, Java, or Scala, and familiarity with SQL.
- Experience with big data technologies like Hadoop, Spark, or Kafka.
- Hands-on experience with cloud platforms, including AWS, Azure, or Google Cloud.
- Strong analytical and problem-solving skills, with the ability to work effectively in a fast-paced environment.
- Excellent communication skills and the ability to collaborate with cross-functional teams.
- Design, implement, and maintain scalable data pipelines and ETL processes.
- Collaborate with team members to architect and optimize data storage solutions.
- Ensure the integrity, availability, and security of data across the entire data ecosystem.
- Assist in the development and implementation of data models to support analytics and reporting.
- Work with cloud technologies, such as AWS, Azure, or Google Cloud, to deploy and manage data solutions.
- Troubleshoot and resolve data-related issues, ensuring data quality and performance.
- Mentor and guide junior data engineering team members with best practices and technical advice.
Posted 2 days ago
ApplyApply
🔥 Senior Data Engineer | Data Delivery
Posted 3 days ago
📍 Germany, Italy, Netherlands, Portugal, Romania, Spain, UK
🧭 Full-Time
🔍 Wellness
- You have a proven track record of designing and building robust, scalable, and maintainable data models and corresponding pipelines from business requirements.
- You are skilled at engaging with engineering and product teams to elicit requirements.
- You are comfortable with big data concepts, ensuring data is efficiently ingested, processed, and made available for data scientists, business analysts, and product teams.
- You are experienced in maintaining data consistency across the entire data ecosystem.
- You have experience maintaining and debugging data pipelines in production environments with high criticality, ensuring reliability and performance.
- Develop and maintain efficient and scalable data models and structures to support analytical workloads.
- Design, develop, and maintain data pipelines that transform and process large volumes of data while embedding business context and semantics.
- Implement automated data quality checks to ensure consistency, accuracy, and reliability of data.
- Ensure correct adoption and usage of Wellhub’s data by data practitioners across the company
- Live the mission: inspire and empower others by genuinely caring for your own wellbeing and your colleagues. Bring wellbeing to the forefront of work, and create a supportive environment where everyone feels comfortable taking care of themselves, taking time off, and finding work-life balance.
SQLApache AirflowKubernetesApache KafkaData engineeringSparkData modeling
Posted 3 days ago
ApplyApply
🔥 Senior Data Engineer | Data Delivery
Posted 3 days ago
📍 Portugal
🧭 Full-Time
🏢 Company: Wellhub
- Proven track record of designing and building robust, scalable, and maintainable data models and corresponding pipelines from business requirements.
- Skilled at engaging with engineering and product teams to elicit requirements.
- Comfortable with big data concepts, ensuring data is efficiently ingested, processed, and made available for data scientists, business analysts, and product teams.
- Experienced in maintaining data consistency across the entire data ecosystem.
- Experience maintaining and debugging data pipelines in production environments with high criticality, ensuring reliability and performance.
- Motivated to contribute to a data-driven culture and take pride in seeing the impact of your work across the company
- Develop and maintain efficient and scalable data models and structures to support analytical workloads.
- Design, develop, and maintain data pipelines that transform and process large volumes of data while embedding business context and semantics.
- Implement automated data quality checks to ensure consistency, accuracy, and reliability of data.
- Ensure correct adoption and usage of Wellhub’s data by data practitioners across the company
- Live the mission: inspire and empower others by genuinely caring for your own wellbeing and your colleagues. Bring wellbeing to the forefront of work, and create a supportive environment where everyone feels comfortable taking care of themselves, taking time off, and finding work-life balance.
SQLApache AirflowETLKubernetesApache KafkaData engineeringSparkData visualizationData modelingData analyticsData management
Posted 3 days ago
ApplyApply
🔥 Semi-Senior Data Engineer (PySpark) - E-Learning
Posted 5 days ago
🧭 Full-Time
🔍 E-Learning
🏢 Company: Truelogic👥 101-250ConsultingWeb DevelopmentWeb DesignSoftware
- 3-5 years of experience working with PySpark and Apache Spark in Big Data environments.
- Experience with SQL and relational and NoSQL databases (PostgreSQL, MySQL, MongoDB, etc.).
- Knowledge of ETL processes and data processing in distributed environments.
- Familiarity with Apache Hadoop, Hive, or Delta Lake.
- Experience with cloud storage (AWS S3, Google Cloud Storage, Azure Blob).
- Proficiency in Git and version control.
- Strong problem-solving skills and a proactive attitude.
- A passion for learning and continuous improvement.
- Design, develop, and optimize data pipelines using PySpark and Apache Spark.
- Integrate and process data from multiple sources (databases, APIs, files, streaming).
- Implement efficient data transformations for Big Data in distributed environments.
- Optimize code to improve performance, scalability, and efficiency in data processing.
- Collaborate with Data Science, BI, and DevOps teams to ensure seamless integration.
- Monitor and debug data processes to ensure quality and reliability.
- Apply best practices in data engineering and maintain clear documentation.
- Stay up to date with the latest trends in Big Data and distributed computing.
Posted 5 days ago
ApplyApply
🔥 Senior Data Engineer, Health & Bioinformatics
Posted 5 days ago
🔍 Health & Bioinformatics
🏢 Company: Gradient AI👥 101-250💰 $20,000,000 Series B almost 4 years agoArtificial Intelligence (AI)Machine LearningInsurTechInsuranceHealth Care
- 5+ years of relevant working experience, with a significant portion focused on healthcare data.
- Proven experience working with and interpreting health, medical, and bioinformatics data is required, including experience with real-world healthcare datasets.
- Expertise as a subject matter expert (SME) in health and bioinformatics data, with a deep understanding of the nuances and challenges associated with processing medical and bioinformatics data, and a strong understanding of the healthcare industry.
- Experience working in Python in a professional environment, ideally in a healthcare or life sciences setting.
- Desire to learn new skills and tools (e.g., Redshift, Tableau, AWS Lambda, etc.); bonus for experience with healthcare-specific data analysis and visualization tools.
- Design, build, and implement data systems that fuel our ML and AI models for our health insurance clients, ensuring compliance with healthcare data privacy and security regulations (e.g., HIPAA).
- Develop tools to extract and process diverse healthcare data sources, including electronic health records (EHRs), medical claims, pharmacy data, and genomic data, and create tools to profile and validate data.
- Work cross-functionally with data scientists to transform large amounts of health-related and bioinformatics data and store it in a format that facilitates modeling, paying close attention to data quality and integrity in the context of healthcare applications.
- Contribute to production operations, data pipelines, workflow management, reliability engineering, and more, with an understanding of the critical nature of data reliability in healthcare settings.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a variety of sources using SQL and AWS ‘big data’ technologies, including experience with healthcare-specific data warehousing and analytics platforms.
- Leverage expertise as a health and bioinformatics SME to ensure that data pipelines align with the specific requirements of health, medical, and bioinformatics data processing, including the ability to translate complex medical and biological concepts into data requirements.
Posted 5 days ago
ApplyApply
🔥 Senior Data Engineer, Health & Bioinformatics
Posted 5 days ago
🔍 Health & Bioinformatics
- BS in Computer Science, Bioinformatics, or another quantitative discipline; 5+ years of relevant working experience, with a significant portion focused on healthcare data.
- Proven experience working with and interpreting health, medical, and bioinformatics data is required, including experience with real-world healthcare datasets.
- Expertise as a subject matter expert (SME) in health and bioinformatics data, with a deep understanding of the nuances and challenges associated with processing medical and bioinformatics data, and a strong understanding of the healthcare industry.
- Experience working in Python in a professional environment, ideally in a healthcare or life sciences setting.
- Desire to learn new skills and tools (e.g., Redshift, Tableau, AWS Lambda, etc.); bonus for experience with healthcare-specific data analysis and visualization tools.
- Design, build, and implement data systems that fuel our ML and AI models for our health insurance clients, ensuring compliance with healthcare data privacy and security regulations (e.g., HIPAA).
- Develop tools to extract and process diverse healthcare data sources, including electronic health records (EHRs), medical claims, pharmacy data, and genomic data, and create tools to profile and validate data.
- Work cross-functionally with data scientists to transform large amounts of health-related and bioinformatics data and store it in a format that facilitates modeling, paying close attention to data quality and integrity in the context of healthcare applications.
- Contribute to production operations, data pipelines, workflow management, reliability engineering, and more, with an understanding of the critical nature of data reliability in healthcare settings.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a variety of sources using SQL and AWS ‘big data’ technologies, including experience with healthcare-specific data warehousing and analytics platforms.
- Leverage expertise as a health and bioinformatics SME to ensure that data pipelines align with the specific requirements of health, medical, and bioinformatics data processing, including the ability to translate complex medical and biological concepts into data requirements.
Posted 5 days ago
ApplyApply
🔥 Senior Data Engineer
Posted 5 days ago
📍 Worldwide
🔍 Hospitality
🏢 Company: Lighthouse
- 4+ years of professional experience using Python, Java, or Scala for data processing (Python preferred)
- You stay up-to-date with industry trends, emerging technologies, and best practices in data engineering.
- Improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed
- Ship large features independently, generate architecture recommendations and have the ability to implement them
- Great communication: Regularly achieve consensus amongst teams
- Familiarity with GCP, Kubernetes (GKE preferred), CI/CD tools (Gitlab CI preferred), familiarity with the concept of Lambda Architecture.
- Experience with Apache Beam or Apache Spark for distributed data processing or event sourcing technologies like Apache Kafka.
- Familiarity with monitoring tools like Grafana & Prometheus.
- Design and develop scalable, reliable data pipelines using the Google Cloud stack.
- Optimise data pipelines for performance and scalability.
- Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
- Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
- Collaborate with the DevOps team to automate deployments and improve developer experience on the data front.
- Work with data science and analytics teams to enable them to bring their research to production grade data solutions, using technologies like airflow, dbt or MLflow (but not limited to)
- As a part of a platform team, you will communicate effectively with teams across the entire engineering organisation, to provide them with reliable foundational data models and data tools.
- Mentor and provide technical guidance to other engineers working with data.
PythonSQLApache AirflowETLGCPKubernetesApache KafkaData engineeringCI/CDMentoringTerraformScalaData modeling
Posted 5 days ago
ApplyApply
🔥 Senior Data Engineer (Kafka/Flink)
Posted 6 days ago
🏢 Company: Workato👥 501-1000💰 $200,000,000 Series E over 3 years ago🫂 Last layoff about 2 years agoSales AutomationCloud ComputingSaaSData IntegrationMarketing Automation
- 5+ years of work experience building & maintaining data pipelines on data-heavy environments (Data Engineering, Backend with emphasis on data processing)
- Fluent knowledge of SQL.
- Strong knowledge of common analytical domain programming languages such as Java, Scala and basic knowledge of Python.
- Strong experience with Flink and Spark.
- Experience with Data Pipeline Orchestration tools (Airflow, Dagster or similar).
- Develop a new usage tracking/billing platform that will provide accurate near real-time data for both circuits.
- Integrate the new platform smoothly with the back office, internal data warehouse, and in-product analytical and reporting tool called Workato Insights.
- Address advanced use cases like usage forecasting, anomaly detection, and real-time alerts.
Posted 6 days ago
ApplyApply
🔥 Senior Data Engineer (Kafka/Flink)
Posted 6 days ago
🔍 Software Development
🏢 Company: Workato👥 501-1000💰 $200,000,000 Series E over 3 years ago🫂 Last layoff about 2 years agoSales AutomationCloud ComputingSaaSData IntegrationMarketing Automation
- 5+ years of work experience building & maintaining data pipelines on data-heavy environments
- Fluent knowledge of SQL
- Strong knowledge of common analytical domain programming languages such as Java, Scala and basic knowledge of Python
- Strong experience with Flink and Spark
- Experience with Data Pipeline Orchestration tools (Airflow, Dagster or similar)
- Experience with Data Warehousing Solutions (Snowflake, Redshift, BigQuery)
- Confidence in using Git, K8s and Terraform
- Develop a new usage tracking/billing platform
- Integrate the platform with back-office, internal data warehouse, and in-product analytical and reporting tool called Workato Insights
- Work closely with the ML team
Posted 6 days ago
ApplyRelated Articles
Posted 8 months ago
Insights into the evolving landscape of remote work in 2024 reveal the importance of certifications and continuous learning. This article breaks down emerging trends, sought-after certifications, and provides practical solutions for enhancing your employability and expertise. What skills will be essential for remote job seekers, and how can you navigate this dynamic market to secure your dream role?
Posted 8 months ago
Explore the challenges and strategies of maintaining work-life balance while working remotely. Learn about unique aspects of remote work, associated challenges, historical context, and effective strategies to separate work and personal life.
Posted 8 months ago
Google is gearing up to expand its remote job listings, promising more opportunities across various departments and regions. Find out how this move can benefit job seekers and impact the market.
Posted 8 months ago
Learn about the importance of pre-onboarding preparation for remote employees, including checklist creation, documentation, tools and equipment setup, communication plans, and feedback strategies. Discover how proactive pre-onboarding can enhance job performance, increase retention rates, and foster a sense of belonging from day one.