Senior Data Engineer

Posted 4 months agoInactiveViewed

View full description

💎 Seniority level: Senior, At least 4+ years

📍 Location: United States, Latin America, India

🔍 Industry: Cloud Data Technologies

🏢 Company: phData👥 501-1000💰 $2,499,997 Seed about 7 years agoInformation Services Analytics Information Technology

🗣️ Languages: English

⏳ Experience: At least 4+ years

🪄 Skills: AWSPythonSoftware DevelopmentSQLElasticSearchGCPHadoopJavaKafkaSnowflakeAirflowAzureCassandraNosqlSparkCommunication SkillsDocumentationScala

Requirements:

At least 4+ years experience as a Software Engineer, Data Engineer or Data Analyst.
Programming expertise in Java, Python and/or Scala.
Core cloud data platforms: Snowflake, AWS, Azure, Databricks, and GCP.
SQL proficiency and the ability to write, debug, and optimize SQL queries.
Experience creating and delivering detailed presentations.

Responsibilities:

Develop end-to-end technical solutions into production.
Ensure performance, security, scalability, and robust data integration.
Client-facing communication and presentation delivery.
Create detailed solution documentation.

Apply

Related Jobs

Apply

🔥 Senior Data Engineer

Posted 5 days ago

📍 Worldwide

🔍 Hospitality

🏢 Company: Lighthouse

🔧 Requirements

4+ years of professional experience using Python, Java, or Scala for data processing (Python preferred)
You stay up-to-date with industry trends, emerging technologies, and best practices in data engineering.
Improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed
Ship large features independently, generate architecture recommendations and have the ability to implement them
Great communication: Regularly achieve consensus amongst teams
Familiarity with GCP, Kubernetes (GKE preferred), CI/CD tools (Gitlab CI preferred), familiarity with the concept of Lambda Architecture.
Experience with Apache Beam or Apache Spark for distributed data processing or event sourcing technologies like Apache Kafka.
Familiarity with monitoring tools like Grafana & Prometheus.

💡 Responsibilities

Design and develop scalable, reliable data pipelines using the Google Cloud stack.
Optimise data pipelines for performance and scalability.
Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
Collaborate with the DevOps team to automate deployments and improve developer experience on the data front.
Work with data science and analytics teams to enable them to bring their research to production grade data solutions, using technologies like airflow, dbt or MLflow (but not limited to)
As a part of a platform team, you will communicate effectively with teams across the entire engineering organisation, to provide them with reliable foundational data models and data tools.
Mentor and provide technical guidance to other engineers working with data.

PythonSQLApache AirflowETLGCPKubernetesApache KafkaData engineeringCI/CDMentoringTerraformScalaData modeling

Posted 5 days ago

Apply

🔥 Senior Data Engineer

Posted 6 days ago

📍 United States

🧭 Full-Time

💸 183600.0 - 216000.0 USD per year

🔍 Software Development

🔧 Requirements

6+ years of experience in a data engineering role building products, ideally in a fast-paced environment
Good foundations in Python and SQL.
Experience with Spark, PySpark, DBT, Snowflake and Airflow
Knowledge of visualization tools, such as Metabase, Jupyter Notebooks (Python)

💡 Responsibilities

Collaborate on the design and improvements of the data infrastructure
Partner with product and engineering to advocate best practices and build supporting systems and infrastructure for the various data needs
Create data pipelines that stitch together various data sources in order to produce valuable business insights
Create real-time data pipelines in collaboration with the Data Science team

PythonSQLSnowflakeAirflowData engineeringSparkData visualizationData modeling

Posted 6 days ago

Apply

🔥 Senior Data Engineer

Posted 6 days ago

📍 United States

🧭 Full-Time

🔍 Healthcare

🏢 Company: Rad AI👥 101-250💰 $60,000,000 Series C 2 months agoArtificial Intelligence (AI)Enterprise Software Health Care

🔧 Requirements

4+ years relevant experience in data engineering.
Expertise in designing and developing distributed data pipelines using big data technologies on large scale data sets.
Deep and hands-on experience designing, planning, productionizing, maintaining and documenting reliable and scalable data infrastructure and data products in complex environments.
Solid experience with big data processing and analytics on AWS, using services such as Amazon EMR and AWS Batch.
Experience in large scale data processing technologies such as Spark.
Expertise in orchestrating workflows using tools like Metaflow.
Experience with various database technologies including SQL, NoSQL databases (e.g., AWS DynamoDB, ElasticSearch, Postgresql).
Hands-on experience with containerization technologies, such as Docker and Kubernetes.

💡 Responsibilities

Design and implement the data architecture, ensuring scalability, flexibility, and efficiency using pipeline authoring tools like Metaflow and large-scale data processing technologies like Spark.
Define and extend our internal standards for style, maintenance, and best practices for a high-scale data platform.
Collaborate with researchers and other stakeholders to understand their data needs including model training and production monitoring systems and develop solutions that meet those requirements.
Take ownership of key data engineering projects and work independently to design, develop, and maintain high-quality data solutions.
Ensure data quality, integrity, and security by implementing robust data validation, monitoring, and access controls.
Evaluate and recommend data technologies and tools to improve the efficiency and effectiveness of the data engineering process.
Continuously monitor, maintain, and improve the performance and stability of the data infrastructure.

AWSDockerSQLElasticSearchETLKubernetesData engineeringNosqlSparkData modeling

Posted 6 days ago

Apply

🔥 Senior Data Engineer - Data Services

Posted 6 days ago

📍 Worldwide

🧭 Full-Time

🔧 Requirements

NOT STATED

💡 Responsibilities

Own the design and implementation of cross-domain data models that support key business metrics and use cases.
Partner with analysts and data engineers to translate business logic into performant, well-documented dbt models.
Champion best practices in testing, documentation, CI/CD, and version control, and guide others in applying them.
Act as a technical mentor to other analytics engineers, supporting their development and reviewing their code.
Collaborate with central data platform and embedded teams to improve data quality, metric consistency, and lineage tracking.
Drive alignment on model architecture across domains—ensuring models are reusable, auditable, and trusted.
Identify and lead initiatives to reduce technical debt and modernise legacy reporting pipelines.
Contribute to the long-term vision of analytics engineering at Pleo and help shape our roadmap for scalability and impact.

SQLData AnalysisETLData engineeringCI/CDMentoringDocumentationData visualizationData modelingData analyticsData management

Posted 6 days ago

Apply

🔥 Senior Data Engineer

Posted 6 days ago

📍 United States

🧭 Full-Time

💸 183600.0 - 216000.0 USD per year

🔍 Mental Healthcare

🏢 Company: Headway👥 201-500💰 $125,000,000 Series C over 1 year agoMental Health Care

🔧 Requirements

6+ years of experience in a data engineering role building products, ideally in a fast-paced environment
Good foundations in Python and SQL.
Experience with Spark, PySpark, DBT, Snowflake and Airflow
Knowledge of visualization tools, such as Metabase, Jupyter Notebooks (Python)
A knack for simplifying data, expressing information in charts and tables

💡 Responsibilities

Collaborate on the design and improvements of the data infrastructure
Partner with product and engineering to advocate best practices and build supporting systems and infrastructure for the various data needs
Create data pipelines that stitch together various data sources in order to produce valuable business insights
Create real-time data pipelines in collaboration with the Data Science team

PythonSQLETLSnowflakeAirflowData engineeringRDBMSSparkRESTful APIsData visualizationData modeling

Posted 6 days ago

Apply

🔥 Senior Data Engineer

Posted 8 days ago

📍 Costa Rica, Brazil, Argentina, Chile, Mexico

🔍 Insider Risk Management

🏢 Company: Teramind👥 51-100 Productivity Tools Security Cyber Security Enterprise Software Software

🔧 Requirements

6+ years of experience in data engineering, with a proven track record of successfully delivering data-driven solutions.
Strong expertise in designing and building scalable data pipelines using industry-standard tools and frameworks.
Experience with big data technologies and distributed systems, such as Hadoop, Spark, or similar frameworks.
Proficient programming skills in languages such as Python, Java, or Scala, alongside a solid understanding of database management systems (SQL and NoSQL).
Understanding of data requirements for machine learning applications and how to optimize data for model training.
Experience with security data processing and compliance standards is preferred, ensuring that data handling meets industry regulations and best practices.

💡 Responsibilities

Design and implement robust data architecture tailored for AI-driven features, ensuring it meets the evolving needs of our platform.
Build and maintain efficient data pipelines for processing user activity data, ensuring data flows seamlessly throughout our systems.
Develop comprehensive systems for data storage, retrieval, and processing, facilitating quick and reliable access to information.
Ensure high standards of data quality and availability, enabling machine learning models to produce accurate and actionable insights.
Enhance the performance and scalability of our data infrastructure to accommodate growing data demands and user activity.
Work closely with data scientists and machine learning engineers to understand their data requirements and ensure data solutions are tailored to their needs.

PythonSQLApache HadoopETLMachine LearningAzureData engineeringNosqlComplianceScalaData visualizationData modelingData management

Posted 8 days ago

Apply

🔥 Senior Data Engineer (AS)

Posted 13 days ago

📍 United States, Canada

🧭 Full-Time

🔍 Software Development

🔧 Requirements

Strong hands-on experience with Python and core Python Data Processing tools such as pandas, numpy, scipy, scikit
Experience with cloud tools and environments like Docker, Kubernetes, GCP, and/or Azure
Experience with Spark/PySpark
Experience with Data Lineage and Data Cataloging
Relational and non-relational database experience
Experience with Data Warehouses and Lakes, such as Bigquery, Databricks, or Snowflake
Experience in designing and building data pipelines that scale
Strong communication skills, with the ability to convey technical solutions to both technical and non-technical stakeholders
Experience working effectively in a fast-paced, agile environment as part of a collaborative team
Ability to work independently and as part of a team
Willingness and enthusiasm to learn new technologies and tackle challenging problems
Experience in Infrastructure as Code tools like Terraform
Advanced SQL expertise, including experience with complex queries, query optimization, and working with various database systems

💡 Responsibilities

Work with business stakeholders to understand their goals, challenges, and decisions
Assist with building solutions that standardize their data approach to common problems across the company
Incorporate observability and testing best practices into projects
Assist in the development of processes to ensure their data is trusted and well-documented
Effectively work with data analysts on refining the data model used for reporting and analytical purposes
Improve the availability and consistency of data points crucial for analysis
Standing up a reporting system in BigQuery from scratch, including data replication, infrastructure setup, dbt model creation, and Integration with reporting endpoints
Revamping orchestration and execution to reduce critical data delivery times
Database archiving to move data from a live database to cold storage

AWSSQLCloud ComputingData AnalysisETLData engineeringData visualizationData modeling

Posted 13 days ago

Apply

🔥 Senior Data Engineer

Posted 20 days ago

📍 Canada, United Kingdom, India

🧭 Full-Time

🔍 Software Development

🏢 Company: Loopio Inc.

🔧 Requirements

5+ years of experience in data engineering in a high-growth agile software development environment
Strong understanding of database concepts, modeling, SQL, query optimization
Ability to learn fast and translate data into actionable results
Experience developing in Python and Pyspark
Hands-on experience with the AWS services (RDS, S3, Redshift, Glue, Quicksight, Athena, ECS)
Strong understanding of relational databases (RDS, MySQL) and NoSQL
Experience with ETL & Data warehousing, building fact & dimensional data models
Experience with data processing frameworks such as Spark / Databricks
Experience in developing Big Data solutions (migration, storage, processing)
Experience with CI/CD tools (Jenkins) and pipeline orchestration tools (Databricks Jobs, Airflow)
Experience working with data visualization and BI platforms (Quicksight, Tableau, Sisense, etc)
Experience working with Clickstream data (Amplitude, Pendo, etc)
Experience building and supporting large-scale systems in a production environment
Strong communication, collaboration, and analytical skills
Demonstrated ability to work with a high degree of ambiguity, and leadership within a team (mentorship, ownership, innovation)
Ability to clearly communicate technical roadmap, challenges, and mitigation

💡 Responsibilities

Be responsible for building, evolving and scaling data platforms and ETL pipelines, with an eye towards the growth of our business and the reliability of our data
Promote data-driven decision-making across the organization through data expertise
Build advanced automation tooling tooling for data orchestration, evaluation, testing, monitoring, administration, and data operations.
Integrate various data sources into our Data lake, including clickstream, relational, and unstructured data
Developing and maintaining a feature store for use in analytics & modeling
Partner with data scientists to create predictive models to help drive insights and decisions, both in Loopio’s product and internal teams (RevOps, Marketing, CX)
Work closely with stakeholders within and across teams to understand the data needs of the business and produce processes that enable a better product and support data-driven decision-making
Build scalable data pipelines using Databricks, and AWS (Redshift, S3, RDS), and other cloud technologies
Build and support Loopio’s data warehouse (Redshift) and data lake (Databricks delta lake)
Orchestrate pipelines using workflow frameworks/tooling

AWSPythonSQLData AnalysisETLJenkinsMachine LearningAirflowData engineeringNosqlSparkCommunication SkillsAnalytical SkillsCollaborationCI/CDData visualizationData modeling

Posted 20 days ago

Apply

🔥 Senior Data Engineer

Posted 23 days ago

📍 United States

🧭 Full-Time

💸 144000.0 - 180000.0 USD per year

🔍 Software Development

🏢 Company: Hungryroot👥 101-250💰 $40,000,000 Series C almost 4 years agoArtificial Intelligence (AI)Food and Beverage E-Commerce Retail Consumer Goods Software

🔧 Requirements

5+ years of experience in ETL development and data modeling
5+ years of experience in both Scala and Python
5+ years of experience in Spark
Excellent problem-solving skills and the ability to translate business problems into practical solutions
2+ years of experience working with the Databricks Platform

💡 Responsibilities

Develop pipelines in Spark (Python + Scala) in the Databricks Platform
Build cross-functional working relationships with business partners in Food Analytics, Operations, Marketing, and Web/App Development teams to power pipeline development for the business
Ensure system reliability and performance
Deploy and maintain data pipelines in production
Set an example of code quality, data quality, and best practices
Work with Analysts and Data Engineers to enable high quality self-service analytics for all of Hungryroot
Investigate datasets to answer business questions, ensuring data quality and business assumptions are understood before deploying a pipeline

AWSPythonSQLApache AirflowData MiningETLSnowflakeAlgorithmsAmazon Web ServicesData engineeringData StructuresSparkCI/CDRESTful APIsMicroservicesJSONScalaData visualizationData modelingData analyticsData management

Posted 23 days ago

Apply

🔥 Senior Data Engineer

Posted 27 days ago

📍 United States

🧭 Full-Time

💸 170000.0 - 190000.0 USD per year

🔍 Software Development

🏢 Company: Productiv👥 101-250💰 $45,000,000 Series C about 4 years agoDeveloper Platform Communities Service Industry SaaS Data Integration Analytics Enterprise Software Software Application Performance Management

🔧 Requirements

Strong experience designing and implementing ETL/ELT data pipelines using modern data stack technologies (e.g., Redshift, Athena, Presto, DynamoDB).
Expertise in data modeling and designing scalable storage solutions for analytics and reporting.
Strong proficiency in SQL, NoSQL, and Javascript
Experience with monitoring, logging, and alerting for data systems to ensure proactive issue resolution.
Experience with data migration to-and-from S3, DynamoDB, Redshift, and Athena.

💡 Responsibilities

Design, build, and maintain scalable, efficient, and reliable data pipelines
Ensure data integrity and quality
Implement monitoring, alerting, and logging systems
Design and optimize data models and storage solutions
Collaborate with cross-functional teams
Continuously improve data engineering processes and standards
Troubleshoot and resolve complex data issues
Mentor and provide technical leadership

AWSSQLApache AirflowDynamoDBETLJavascriptCross-functional Team LeadershipAlgorithmsData engineeringData StructuresREST APINosqlCommunication SkillsCI/CDProblem SolvingMentoringData visualizationData modelingSoftware EngineeringData management

Posted 27 days ago

Apply