Staff Software Engineer, Data Platform
New
A
Agility RoboticsRobotics
All of our roles are U.S.-based.Full-TimeStaff
Salary$191,000 — $299,000 USD
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- AWSPythonGCPJavaKafkaAzurePrometheusSparkScala
Requirements
- 5+ years of strong experience in building and scaling cloud-based data platforms with familiarity in cloud providers like AWS, GCP or Azure
- Proficient in big data frameworks like Spark, Kafka, Beam, Flink etc
- Familiarity with OSS data formats like Parquet, Arrow, Iceberg and protocols like Avro and Protobuf
- Experience with observability tools (e.g. OpenTelemetry, Prometheus etc) and implementing metrics, tracing and logging systems
- Proficiency in languages like Java, Scala (any JVM), or Python
- Solid understanding of distributed systems and microservices architectures
Responsibilities
- Design and implement scalable data pipelines for ingesting and processing telemetry, logs and sensor data from our fleet of robots and associated cloud software
- Develop and maintain data ingestion agents running directly on robot software to ingest high volume data in low connectivity environments
- Maintain and develop data formats and protocols optimized for transmission, storage and query of data recordings, sensor data and telemetry
- Develop and maintain observability systems using tools like fluentbit, OpenTelemetry etc enabling real-time monitoring and analytics of fleet operations
- Implement solutions to prepare high volume, high velocity data for analysis using technologies such as Spark, Kafka or equivalent
- Build and optimize tools for internal stakeholders for business intelligence and long-horizon fleet analytics and forecasting using existing tools like AWS S3, Athena, Sagemaker etc
- Leverage IoT experience to improve data collection and processing pipelines from robot sensors and edge devices
View Full Description & ApplyYou'll be redirected to the employer's site
