Data Engineer

Posted 2024-11-19

View full description

💎 Seniority level: Middle, 4-6 years

📍 Location: Germany

🏢 Company: Axiom Software Solutions Limited

🗣️ Languages: German, English

⏳ Experience: 4-6 years

🪄 Skills: ETLMachine LearningData engineeringRDBMSNosql

Requirements:

4-6 years of experience in data architecture and engineering roles.
3+ years of hands-on experience with relational, dimensional, and/or analytic technologies.
Knowledge of Sagemaker/Jupyter and data lake.
Experience with data warehouse, data lake, and big data platforms.
Implement business and IT data requirements through new data strategies and designs.
Identify architecture, infrastructure, and interfaces for automated data loads and security concerns.

Responsibilities:

Collaborate with stakeholders to understand and document data requirements, business rules, and objectives for the data platform.
Design and develop conceptual, logical, and physical data models that accurately represent the organization’s data assets.
Ensure designs meet objectives for reliability, scalability, supportability, user experience, security, governance, and performance.
Implement data engineering practices for data integrity, performance, and scalability.
Work closely with architects to integrate data engineering into overall platform architecture.
Communicate design decisions and recommendations effectively with stakeholders.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer - Search

Posted 2024-11-26

📍 US, Germany, UK

🧭 Full-Time

🔍 Music streaming

🏢 Company: SoundCloud

Strong background in data engineering with at least 5 years of experience.
Experience in designing and optimizing data pipelines, data architecture and modeling, and ETL processes for large-scale distributed systems.
Proficient in SQL and Python, ideally with experience in Scala or Java.
Experience in training, prototyping, and deploying machine learning models.
Experience in Search or Recommender Systems is a plus.

Build and optimize high-end data pipelines that drive key performance indicators (KPIs) and train Machine Learning models.
Work on real-time processing of data pipelines for search features used by millions of users.
Collaborate with cross-functional teams to ensure structured data for search systems and machine learning models.

PythonSQLAgileETLMachine LearningAlgorithmsData engineeringData StructuresSpark

Posted 2024-11-26

Apply

🔥 Senior Data Engineer

Posted 2024-11-22

📍 United States, United Kingdom, Singapore, Indonesia, Germany, France, Japan, Australia

🔍 Customer engagement platform

🏢 Company: Braze👥 1001-5000💰 $ Grant on 2023-06-21CRM Analytics Marketing Marketing Automation Software

5+ years of hands-on experience in data engineering, cloud data warehouses, and ETL development.
Proven expertise in designing and optimizing data pipelines and architectures.
Strong proficiency in advanced SQL and data modeling techniques.
Experience leading impactful data projects from conception to deployment.
Effective collaboration skills with cross-functional teams and stakeholders.
In-depth understanding of technical architecture and data flow in a cloud-based environment.
Ability to mentor and guide junior team members.
Passion for building scalable data solutions.
Strong analytical and problem-solving skills with attention to detail.
Experience with large event-level data aggregation.
Familiarity with data governance principles.

Lead the design, implementation, and monitoring of scalable data pipelines and architectures using tools like Snowflake and dbt.
Develop and maintain robust ETL processes to ensure high-quality data ingestion, transformation, and storage.
Collaborate with data scientists, analysts, and engineers to implement data solutions for customer engagement.
Optimize and manage data flows across various platforms and applications.
Ensure data quality, consistency, and governance through best practices.
Work with large-scale event-level data to support business intelligence and analytics.
Implement and maintain data products using advanced techniques.
Collaborate with cross-functional teams to deliver valuable data solutions.
Evaluate and integrate new data technologies to enhance data infrastructure.

SQLBusiness IntelligenceETLSnowflakeData engineeringCollaborationCompliance

Posted 2024-11-22

Apply

🔥 Senior Data Engineer - Data Corpus

Posted 2024-11-07

📍 US, Germany, UK

🧭 Full-Time

🔍 Music

🏢 Company: SoundCloud

Senior Level Data Professional with a minimum of 4 years of experience (ideal 6+ years).
Experience with Cloud technologies, specifically GCP (required), with AWS/Azure as a plus.
Experience working with BigQuery and advanced SQL knowledge.
Proficiency in Python and Airflow.
Experience with big data at terabyte/petabyte scale.
Data Architecture/solution design experience.
Familiarity with Agile methodology and Jira.
Experience in data warehousing and analytical data modeling.
Knowledge of CI/CD pipelines and Git.
Experience in building reliable ETL pipelines and datasets for BI tools (Looker preferred).
Basic statistical knowledge and ability to produce high-quality technical documentation.

Build and maintain a unified and standardized data warehouse, Corpus, at SoundCloud.
Abstract the complexity of SoundCloud’s vast data ecosystem.
Collaboration with business reporting, data science, and product teams.
Gather and refine requirements, design data architecture and solutions.
Build ETL pipelines using Airflow to land data in BigQuery.
Model and build the business-ready data layer for dashboarding tools.

PythonSQLAgileETLGCPGitJiraAirflowCI/CD

Posted 2024-11-07

Apply

🔥 Senior Data Engineer (AI), USA

Posted 2024-10-17

📍 USA, UK, Germany

💸 $70,000 - $205,000 per year

🔍 Cybersecurity

🏢 Company: Cobalt

Minimum of 5 years experience in data engineering with a strong background in Google BigQuery, Looker Studio, and DBT.
Expertise in Terraform, Python, and SQL for data transformation and infrastructure management.
Excellent verbal and written communication skills in English, enabling effective collaboration in a remote setting.
Eagerness to learn new technologies and approaches, with a proactive mindset and willingness to contribute ideas.
Understanding of Machine Learning and Generative AI.

Design, build, and maintain scalable and robust data pipelines in BigQuery, ensuring data integrity and efficiency.
Empower finance, marketing and product with data as well as providing business with valuable data insights.
Collaborate closely with Software Engineers to integrate Generative AI and Large Language Models into our data systems, focusing on automation and advanced analytics.
Manage our data lake and warehouse to also support AI and ML initiatives.
Utilize Terraform for infrastructure as code and develop Python applications for data importation, event-triggering processes, MLOps and more.
Work with various teams to understand data requirements and deliver insights and solutions that drive decision-making and product innovation.

PythonSQLCybersecurityMachine LearningData engineeringCommunication SkillsCollaborationTerraform

Posted 2024-10-17

Apply

🔥 Data Engineer, Data Services

Posted 2024-08-10

📍 Central EU or Americas

🧭 Full-Time

🔍 Real Estate Investment

🏢 Company: Roofstock👥 501-1000💰 $240.0m Series E on 2022-03-10🫂 on 2023-03-22Rental Property PropTech Marketplace Real Estate FinTech

BS or MS in a technical field: computer science, engineering or similar.
5+ years technical experience working with data.
5+ strong experience building scalable data services and applications using either SQL, Python, Java / Kotlin.
Deep understanding of microservices architecture and RESTful API development including gRPC, REST/SOAP, GraphQL.
Experience with AWS services including Messaging such as SQS, SNS, and familiarity with real-time data processing frameworks.
Significant experience building and deploying data-related infrastructure, robust data pipelines, and ETL/ELT code.
Strong understanding of data architecture and related problems.
Experience working on complex problems and distributed systems where scalability and performance are important.
Strong communication and interpersonal skills.
Independent work and effective collaboration with cross-functional teams.

Improve and maintain the data services platform.
Deliver high-quality data services promptly, ensuring data governance and integrity while meeting objectives and maintaining SLAs for data sharing across multiple products.
Develop effective architectures and produce key code components that contribute to the design, implementation, and maintenance of technical solutions.
Integrate a diverse network of third-party tools into a cohesive, scalable platform, optimizing code for enhanced scalability, performance, and readability.
Continuously improve system performance and reliability by diagnosing and resolving unexpected operational issues.
Ensure the team's work undergoes rigorous testing through repeatable, automated methods.
Support the data infrastructure and the rest of the data team in designing, implementing, and deploying scalable, fault-tolerant pipelines.

AWSDockerGraphQLPythonSQLAgileETLJavaKafkaKotlinSCRUMSnowflakeAirflowApache KafkaData engineeringgRPC

Posted 2024-08-10

Apply

🔥 Senior Data Engineer, Data Services

Posted 2024-08-10

📍 Central EU or Americas

🧭 Full-Time

🔍 Real estate investment

🏢 Company: Roofstock👥 501-1000💰 $240.0m Series E on 2022-03-10🫂 on 2023-03-22Rental Property PropTech Marketplace Real Estate FinTech

BS or MS in a technical field: computer science, engineering or similar.
8+ years technical experience working with data.
5+ years strong experience building scalable data services and applications using SQL, Python, Java/Kotlin.
Deep understanding of microservices architecture and RESTful API development.
Experience with AWS services including messaging and familiarity with real-time data processing frameworks.
Significant experience building and deploying data-related infrastructure and robust data pipelines.
Strong understanding of data architecture and related challenges.
Experience with complex problems and distributed systems focusing on scalability and performance.
Strong communication and interpersonal skills.
Independent worker able to collaborate with cross-functional teams.

Improve and maintain the data services platform.
Deliver high-quality data services promptly, ensuring data governance and integrity while meeting objectives and maintaining SLAs.
Develop effective architectures and produce key code components contributing to technical solutions.
Integrate a diverse network of third-party tools into a cohesive, scalable platform.
Continuously enhance system performance and reliability by diagnosing and resolving operational issues.
Ensure rigorous testing of the team's work through automated methods.
Support data infrastructure and collaborate with the data team on scalable data pipelines.
Work within an Agile/Scrum framework with cross-functional teams to deliver value.
Influence the enterprise data platform architecture and standards.

AWSDockerPythonSQLAgileETLSCRUMSnowflakeAirflowData engineeringgRPCRESTful APIsMicroservices

Posted 2024-08-10

Apply

🔥 Data Engineer

Posted 2024-07-18

📍 Africa, United Kingdom, Europe, Middle East

🧭 Full-Time

🔍 Sports and Digital Entertainment

4+ years of experience in a data engineering or similar role.
Excellent programming skills in Python and Spark (PySpark / Databricks).
2+ years' experience with Databricks and Azure data services.
Experience with other cloud-based data management environments (AWS, Google Cloud, etc.) is an advantage.
Experience working with Customer Data Platforms is a plus.
Knowledge of managing data quality, including monitoring and alerting.
Good understanding of application and database development lifecycles.
Experience with remote working and ideally with hyper-growth startups.

Building and managing a highly robust and scalable Data Lake/ETL infrastructure.
Creating a scalable data pipeline for streaming and batch processing.
Ensuring data integrity through fault-tolerant systems and automated data quality monitoring.
Continuously improving processes and optimizing performance and scalability.
Ensuring privacy and data security are prioritized.
Documenting the Data Platform stack comprehensively.
Partnering with business stakeholders and product engineering to deliver data products.
Collaborating with stakeholders to shape requirements and drive the data platform roadmap.

PythonAgileETLAzureData engineeringSparkDocumentation

Posted 2024-07-18

Apply

🔥 Data Engineer

Posted 2024-07-18

📍 Africa, United Kingdom, Europe, Middle East

🔍 Sports and Digital Entertainment

4+ years' of experience in a data engineering or similar role.
Excellent programming skills in Python and Spark (PySpark/Databricks).
2+ years’ experience working with Databricks and Azure data services.
Experience with other cloud data environments (AWS, Google Cloud, Hadoop, etc.) is a plus.
Experience with Customer Data Platforms is advantageous.
Active management of data quality including monitoring and alerting.
Familiarity with the application lifecycle from development to production.
Remote working experience is required; hyper-growth startup experience is a plus.

Design and manage a scalable Data Lake/ETL infrastructure and data pipeline.
Ensure fault-tolerant systems and processes with data integrity and automated monitoring.
Continuously improve data processes for performance and scalability.
Ensure data privacy and security standards are met.
Maintain up-to-date documentation for the Data Platform stack.
Collaborate with business stakeholders and engineering to deliver data products.

PythonAgileETLKafkaAzureData engineeringSpark

Posted 2024-07-18

Apply

🔥 Data Engineer

Posted 2024-07-18

📍 Africa, United Kingdom, Europe, Middle East

🧭 Full-Time

🔍 Sports and Digital Entertainment

4+ years' of experience in a data engineering or similar role.
Excellent programming language skills in Python and Spark (PySpark / Databricks).
2+ years’ experience working with Databricks and Azure data services.
Experience with cloud-based data management environments like AWS, Google Cloud, Hadoop, Snowflake, Spark, Storm, Kafka is an advantage.
Experience managing data quality including monitoring and alerting.
Good understanding of the application and database development lifecycle.
Remote working experience is a must; hyper-growth startup experience is a strong plus.

Design and manage a highly robust and scalable Data Lake/ETL infrastructure and scalable data pipelines for streaming and batch processing.
Ensure fault-tolerant systems and processes with a high priority on data integrity, supported by automated quality monitoring and alerting.
Continuously seek improvements through fixing recurring problems, delivering helpful features, and optimizing for performance and scalability.
Maintain top-class and up-to-date documentation for the entire Data Platform stack.
Partner with business stakeholders and product engineering to deliver high-value data products and understand requirements.

PythonETLAzureData engineeringSpark

Posted 2024-07-18

Apply

Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities