Staff Software Engineer, Data
New
J
JobgetherData infrastructure
Based in the United StatesFull-TimeStaff
Salary$175,500–$230,000 USD for eligible candidates hired in the United States
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 6 years of experience building and operating production data platforms, including responsibility for architectural decisions.
- Required Skills
- AWSPythonSQLKubernetesSparkTerraformdbtHelm
Requirements
- Have at least 6 years of experience building and operating production data platforms, including architectural decision-making.
- Have experience integrating SaaS business systems such as Stripe, HubSpot, ERP platforms, and identity providers into a centralized data warehouse.
- Have production experience preparing large-scale multimodal training datasets and running batch workloads using HPC or schedulers such as SLURM or Kubernetes.
- Demonstrate expert-level SQL and Python skills.
- Use automated testing, code reviews, and continuous integration as part of strong software engineering practices.
- Have hands-on experience with data warehouses and data lakes, data modeling, and technologies such as Apache Iceberg or Delta Lake on S3, Athena or Trino, Snowflake or BigQuery, and dbt.
- Have experience operating orchestration systems such as dbt, Airflow, or Argo and distributed processing technologies such as Spark or equivalent frameworks.
- Have strong knowledge of AWS, Kubernetes and EKS, Helm, and Terraform, including troubleshooting data-processing jobs and infrastructure deployments.
- Work autonomously, make technical decisions, and take end-to-end responsibility for production systems.
- Use analytical judgment to interpret data, communicate its limitations, and clarify what metrics represent.
- Valued additional experience includes building a data platform from the ground up, collaborating with generative AI research teams, FinOps and cloud cost reporting, and SOC 2 or similar compliance frameworks.
Responsibilities
- Own and maintain the data platform, including the AWS data lake, dbt transformation layer, ingestion pipelines, and PostgreSQL serving layer.
- Build and maintain integrations across billing, CRM, ERP, identity, and telemetry systems.
- Develop warehouse models, dimensional schemas, product event data contracts, data quality tests, lineage tracking, and catalog documentation.
- Implement PII access controls and data classification practices with Security and Legal teams.
- Optimize Iceberg tables, partitioning, query performance, and infrastructure costs for large telemetry and log datasets.
- Manage data orchestration and infrastructure on Kubernetes and Amazon EKS using Helm, Terraform, CI/CD, monitoring, and alerting.
- Troubleshoot production data pipeline failures as part of the business-hours on-call rotation.
- Deliver operational reporting for Finance and leadership on revenue, credits, liabilities, and cloud spending.
- Source, verify licenses for, ingest, and version large training datasets across audio, video, image, and 3D modalities.
- Build media-generation, filtering, deduplication, and captioning pipelines and maintain dataset lineage, quality metrics, and documentation.
- Lead technical design reviews, mentor teammates, and develop operational runbooks.
View Full Description & ApplyYou'll be redirected to the employer's site
