Senior Data Engineer, Data Analytics
New
A
Agility RoboticsRobotics data
All of our roles are U.S.-based.Full-TimeSenior
Salary$175,000 — $274,000 USD
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of experience as a Data Engineer or similar role building and maintaining production data pipelines.
- Required Skills
- AWSPythonJavaAirflowSparkScalaData modeling
Requirements
- Have 5+ years of experience as a Data Engineer or in a similar role building and maintaining production data pipelines.
- Demonstrate strong proficiency with Apache Spark or an equivalent distributed data processing framework.
- Have complementary programming skills in Python, Java, and/or Scala.
- Have experience with Airflow, Dagster, Prefect, or other data orchestration systems.
- Be proficient with Avro, Parquet, and structured or numeric datasets.
- Be familiar with table formats such as Apache Iceberg or Delta Lake and medallion architecture.
- Understand data modeling, schema evolution, and data quality best practices.
- Know how to model datasets logically and partition them physically for query performance, including analytical queries and playback or root-cause analysis.
- Work cross-functionally with software, hardware, and analytics teams.
- Nice to have experience with vision data pipelines, C++, data versioning or immutable dataset design, and data catalog systems.
Responsibilities
- Collaborate with robot software and hardware teams to define, collect, and curate data for analytics and debugging.
- Develop and maintain ETL pipelines that transform raw robot logs and telemetry into structured datasets using Spark, orchestration tools, and AWS data services.
- Own gold-layer datasets for fleet dashboards, reliability models, and business metrics, with freshness SLAs, documentation, and monitoring.
- Implement data quality checks, schema evolution, and metadata management using internal data registry and cataloging systems.
- Collaborate with reliability, analytics, and ML teams to design data models for their workflows.
- Contribute to shared libraries or APIs in Python, Java, or C++ to support data capture and consumption.
View Full Description & ApplyYou'll be redirected to the employer's site
