Senior Data Engineer

Posted 2 months agoViewed

💎 Seniority level: Senior, 3+ years

📍 Location: United States

⏳ Experience: 3+ years

🪄 Skills: LeadershipPythonSoftware DevelopmentSQLGitMachine LearningMLFlowStrategyAzureData engineeringSparkCollaborationCI/CD

Requirements:

3+ years in data engineering roles in a production environment.
Advanced proficiency in Python and SQL for data engineering.
Up-to-date knowledge of and 1+ years of experience using Databricks for Lakehouse management.
Deep understanding of data modeling, data architecture, and data integration best practices.
Strong hands-on experience with Apache Spark.
Familiarity with data governance, security, and privacy principles.
Comfort using git or equivalent to manage the software development life cycle.
Exceptional ability to learn and use new software development techniques and tools.
Ability to manage multiple projects simultaneously.
High energy, humble team player with 'get it done' attitude, seeking collaboration with colleagues.

Responsibilities:

Collaborate with cross-functional teams to define the data engineering strategy aligned to business objectives, including data modeling that unifies data assets across a range of source systems used to manage the operations of our partnering hospitals.
Define and execute processes needed to develop, test, deploy, and maintain high quality data pipelines. Oversee the end-to-end development of data pipelines from source data extraction through to production-grade analytical dataset delivery, ensuring data quality and security throughout the pipeline.
Continuously monitor and optimize data processing performance and efficiency. Identify and address bottlenecks, optimize query performance, and improve overall system stability.
Establish and enforce data quality management policies, data access controls, and data privacy standards.
Stay abreast of the latest developments in engineering tools and best practices. Provide guidance to the team about technical challenges.
Maintain clear and comprehensive documentation of data pipelines, architecture, and processes to ensure knowledge sharing and team continuity.
Evaluate and manage relationships with third-party vendors and tools, making informed decisions about when to leverage external solutions.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer (Remote, USA)

Posted 3 days ago

📍 USA

💸 152960.0 - 183552.0 USD per year

🔍 Data Engineering and Observability Solutions

Software development skills in Python, Java, Scala, or Go.
High proficiency in SQL.
Experience with workflow orchestration systems like Prefect, Dagster, or Airflow.
Knowledge of MLOps best practices.
Familiarity with dbt or similar data transformation tools.
Excellent communication skills for technical topics.

Build and maintain production quality data pipelines between operational systems and BigQuery.
Implement data quality and freshness checks to ensure data accuracy and consistency.
Build and maintain machine learning pipelines for automated model validation and deployment.
Create and maintain documentation for data engineering processes and workflows.
Maintain observability and monitoring of internal data pipelines.
Troubleshoot data pipeline issues to ensure data availability.
Contribute to dbt systems ensuring efficiency and availability.

PythonSQLETLGCPMachine LearningData engineering

Posted 3 days ago

Apply

🔥 Senior Data Engineer (MLOps)

Posted 6 days ago

📍 U.S., U.K., European Union

🔍 Martech

5+ years of experience in data engineering or a related field.
Strong expertise in data pipeline orchestration tools such as Apache Airflow.
Proven track record of designing and implementing data lakes and warehouses, preferably with Azure.
Solid understanding of MLOps practices.
Proficiency in programming languages such as Python and SQL.
Experience with distributed computing frameworks like Spark.
Familiarity with version control systems like Git and CI/CD pipelines.
Effective communication skills and ability to collaborate with cross-functional teams.

Design and implement scalable, secure data lake and warehouse solutions.
Develop, monitor, and optimize ETL/ELT workflows using Apache Airflow.
Develop scalable web scraping solutions and process unstructured data.
Design cloud-native data solutions with Microsoft Azure.
Use Terraform for infrastructure automation.
Collaborate with data scientists on MLOps and implement CI/CD pipelines.
Utilize Kubernetes and Docker for deploying data processing systems.
Collaborate with stakeholders to provide solutions and maintain documentation.

DockerPythonSQLApache AirflowKubernetesMicrosoft AzureData engineeringSparkTerraform

Posted 6 days ago

Apply

🔥 Senior Data Engineer

Posted 6 days ago

📍 Denver, CO

🧭 Full-Time

🔍 Construction

🏢 Company: EquipmentShare👥 1001-5000💰 $400,000,000 Debt Financing over 1 year agoConstruction

7+ years of relevant data platform development experience building production-grade solutions.
Proficient with SQL and a high-order object-oriented language (e.g., Python).
Experience with designing and building distributed data architecture.
Experience building and managing production-grade data pipelines using tools such as Airflow, dbt, DataHub, MLFlow.
Experience building and managing production-grade data platforms using distributed systems such as Kafka, Spark, Flink and/or others.
Familiarity with event data streaming at scale.
Proven track record learning new technologies and applying that learning quickly.
Experience building observability and monitoring into data products.
Motivated to identify opportunities for automation to reduce manual toil.

Collaborate with Product Managers, Designers, Engineers, Data Scientists, and Data Analysts to take ideas from concept to production at scale.
Design, build and maintain a data platform to enable automation and self-service for data scientists, machine learning engineers, and analysts.
Design, build and maintain data product framework to support EquipmentShare application data science and analytics features.
Design, build and maintain CI/CD pipelines and automated data and machine learning deployment processes.
Develop data monitoring and alerting capabilities.
Document architecture, processes, and procedures for knowledge sharing and cross-team collaboration.
Mentor peers to help them build their skills.

AWSPythonSQLApache AirflowKafkaMLFlowSnowflakeSparkCI/CD

Posted 6 days ago

Apply

🔥 Senior Data Engineer

Posted 11 days ago

📍 United States, United Kingdom, Spain, Estonia

🔍 Identity verification

🏢 Company: Veriff👥 501-1000💰 $100,000,000 Series C almost 3 years ago🫂 Last layoff over 1 year agoArtificial Intelligence (AI)Fraud Detection Information Technology Cyber Security Identity Management

Expert-level knowledge of SQL, particularly with Redshift.
Strong experience in data modeling with an understanding of dimensional data modeling best practices.
Proficiency in data transformation frameworks like dbt.
Solid programming skills in languages used in data engineering, such as Python or R.
Familiarity with orchestration frameworks like Apache Airflow or Luigi.
Experience with data from diverse sources including RDBMS and APIs.

Collaborate with business stakeholders to design, document, and implement robust data models.
Build and optimize data pipelines to transform raw data into actionable insights.
Fine-tune query performance and ensure efficient use of data warehouse infrastructure.
Ensure data reliability and quality through rigorous testing and monitoring.
Assist in migrating from batch processing to real-time streaming systems.
Expand support for various use cases including business intelligence and analytics.

PythonSQLApache AirflowETLData engineeringJSONData modeling

Posted 11 days ago

Apply

🔥 Senior Data Engineer

Posted 11 days ago

📍 TX, MN, FL

💸 130000.0 - 195000.0 USD per year

🔍 Healthcare

🏢 Company: NeueHealth

Bachelor’s degree in Computer Science, Computer Engineering, Information Systems, or equivalent.
Around five years of experience in an enterprise data engineering role in an Azure environment.
Healthcare IT background preferred.
Experience coding in Scala and building batch and streaming data pipelines.
Experience with API design.
Extensive experience developing data solutions in Azure Cloud.
Experience with event sourcing and/or Big Data architectures.

Write traditional code and server-less functions mainly in Scala.
Build APIs, data microservices, and ETL pipelines for data sharing and analytics.
Develop and optimize processes for large language models and AI enhancements.
Support Data Ingestion frameworks deployed in Azure.
Participate in cultivating a culture of DevOps and Quality Assurance.
Act as tech lead and mentor junior engineers.
Continuously document code and team processes.

ETLC#AzureSparkCI/CDDevOpsMicroservicesScala

Posted 11 days ago

Apply

🔥 Senior Data Engineer

Posted 12 days ago

📍 USA

🧭 Full-Time

💸 165000.0 - 210000.0 USD per year

🔍 E-commerce and AI technologies

🏢 Company: Wizard👥 11-50 Customer Service Manufacturing

5+ years of professional experience in software development with a focus on data engineering.
Bachelor's degree in Computer Science or a related field, or equivalent practical experience.
Proficiency in Python with software engineering best practices.
Strong expertise in building ETL pipelines using tools like Apache Spark.
Hands-on experience with NoSQL databases like MongoDB, Cassandra, or DynamoDB.
Proficiency in real-time stream processing systems such as Kafka or AWS Kinesis.
Experience with cloud platforms (AWS, GCP, Azure) and technologies like Delta Lake and Parquet files.

Develop and maintain scalable data infrastructure for batch and real-time processing.
Build and optimize ETL pipelines for efficient data flow.
Collaborate with data scientists and cross-functional teams for accurate monitoring.
Design backend data solutions for microservices architecture.
Implement and manage integrations with third-party e-commerce platforms.

AWSPythonDynamoDBElasticSearchETLGCPGitHadoopKafkaMongoDBRabbitmqAzureCassandraRedis

Posted 12 days ago

Apply

🔥 Senior Data Engineer

Posted 14 days ago

📍 United States

🧭 Full-Time

🏢 Company: Avalore, LLC

Master’s or PhD in statistics, mathematics, computer science, or related field.
8+ years of experience as a Data Engineer within the IC.
Outstanding communication skills, influencing abilities, and client focus.
Professional proficiency in English is required.
Current, active Top Secret security clearance.
Applicants must be currently authorized to work in the United States on a full-time basis.

Develops and documents data pipelines for ingest, transformation, and preparation of data for AI applications.
Designs scalable technologies such as streaming and transformation, joining disparate data sets for predictive analytics.
Develops API interfaces for accessibility.
Leads technical efforts and guides development teams.

PythonSQLApache AirflowArtificial IntelligenceETLMachine LearningAPI testingData engineering

Posted 14 days ago

Apply

🔥 Senior Data Engineer

Posted 16 days ago

📍 USA

🧭 Full-Time

💸 190000.0 - 220000.0 USD per year

🔍 B2B data / Data as a Service (DaaS)

🏢 Company: People Data Labs👥 101-250💰 $45,000,000 Series B about 3 years agoDatabase Artificial Intelligence (AI)Developer APIs Machine Learning Analytics B2B Software

5-7+ years industry experience with strategic technical problem-solving.
Strong software development fundamentals.
Experience with Python.
Expertise in Apache Spark (Java, Scala, or Python-based).
Proficiency in SQL.
Experience building scalable data processing systems.
Familiarity with data pipeline orchestration tools (e.g., Airflow, dbt).
Knowledge of modern data design and storage patterns.
Experience working in Databricks.
Familiarity with cloud computing services (e.g., AWS, GCP, Azure).
Experience in data warehousing technologies.
Understanding of modern data storage formats and tools.

Build infrastructure for ingestion, transformation, and loading of data using Spark, SQL, AWS, and Databricks.
Create an entity resolution framework for merging billions of entities into clean datasets.
Develop CI/CD pipelines and anomaly detection systems to enhance data quality.
Provide solutions to undefined data engineering problems.
Assist Engineering and Product teams with data-related technical issues.

AWSPythonSQLKafkaAirflowData engineeringPandasCI/CD

Posted 16 days ago

Apply

🔥 Senior Data Engineer

Posted about 1 month ago

📍 United States, United Kingdom, Singapore, Indonesia, Germany, France, Japan, Australia

🔍 Customer engagement platform

🏢 Company: Braze👥 1001-5000💰 Grant over 1 year agoCRM Analytics Marketing Marketing Automation Software

5+ years of hands-on experience in data engineering, cloud data warehouses, and ETL development.
Proven expertise in designing and optimizing data pipelines and architectures.
Strong proficiency in advanced SQL and data modeling techniques.
Experience leading impactful data projects from conception to deployment.
Effective collaboration skills with cross-functional teams and stakeholders.
In-depth understanding of technical architecture and data flow in a cloud-based environment.
Ability to mentor and guide junior team members.
Passion for building scalable data solutions.
Strong analytical and problem-solving skills with attention to detail.
Experience with large event-level data aggregation.
Familiarity with data governance principles.

Lead the design, implementation, and monitoring of scalable data pipelines and architectures using tools like Snowflake and dbt.
Develop and maintain robust ETL processes to ensure high-quality data ingestion, transformation, and storage.
Collaborate with data scientists, analysts, and engineers to implement data solutions for customer engagement.
Optimize and manage data flows across various platforms and applications.
Ensure data quality, consistency, and governance through best practices.
Work with large-scale event-level data to support business intelligence and analytics.
Implement and maintain data products using advanced techniques.
Collaborate with cross-functional teams to deliver valuable data solutions.
Evaluate and integrate new data technologies to enhance data infrastructure.

SQLBusiness IntelligenceETLSnowflakeData engineeringCollaborationCompliance

Posted about 1 month ago

Apply

🔥 Senior Data Engineer

Posted about 1 month ago

📍 USA

🧭 Full-Time

💸 140000 - 160000 USD per year

🔍 Health tech

🏢 Company: Carrum Health

10+ years professional experience as a data engineer, including ownership of data products.
Proficiency with data engineering technologies including Python, PostgreSQL, AWS Athena, and Docker.
History of designing systems focused on data quality and scalability.
Experience in the healthcare space or another highly-regulated industry.
Strong interpersonal skills and the ability to work collaboratively.

Develop and maintain creative solutions to evolving challenges that span data ingestion, processing, and modeling.
Collaborate closely with internal and external stakeholders to understand data needs.
Grow and maintain pipelines that power analytics, machine learning, and data products.
Participate early in the development of data foundation and infrastructure, implementing quality control and reporting systems.
Understand HIPAA compliance and support Carrum’s success in this area.

AWSDockerPostgreSQLPythonBashETLMachine LearningRubyTableauData engineeringData scienceDevOpsCompliance

Posted about 1 month ago

Apply

Senior Data Engineer

Requirements:

Responsibilities:

Related Jobs

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities

🔧 Requirements

💡 Responsibilities