Sr. Principal Data Engineer / Data Architect
United States (Remote), multiple geographies and time zonesFull-TimePrincipal
Salary131,250 - 175,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 10+ years in Data Engineering, with at least 3 years in a Principal or Architect capacity. Minimum 4+ years of hands-on experience specifically within the Databricks ecosystem
- Required Skills
- PythonSQLGCPSparkScalaBigQueryDatabricksPySpark
Requirements
- Minimum 4+ years of hands-on experience specifically within the Databricks ecosystem (Delta Lake, Unity Catalog, Photon).
- Deep proficiency in Spark (PySpark/Scala), Python, and SQL.
- Experience with GCP (BigQuery) is a major plus.
- Proven experience managing petabyte-scale datasets and high-concurrency pipelines (AdTech experience preferred).
- Must be a self-starter capable of taking a high-level concept from the SVP and driving it to completion with minimal supervision.
- 10+ years in Data Engineering, with at least 3 years in a Principal or Architect capacity.
- A history of successfully diagnosing and resolving complex, large-scale production bottlenecks and data outages.
- A demonstrated ability to multi-task, balancing long-term architectural R&D with immediate business requests and partner integrations.
- Strong understanding of optimizing Databricks/BigQuery costs and the ability to communicate technical trade-offs to non-technical stakeholders.
Responsibilities
- Design and oversee the migration/optimization of our data lakehouse architecture using Databricks.
- Serve as the escalation point for the platform, leveraging a proven record of handling production issues to ensure 24/7 reliability of mission-critical pipelines.
- Work directly with the SVP of Tech to translate product roadmaps into technical requirements.
- Drive the adoption of Databricks Asset Bundles (DABs) to standardize deployments across dev, staging, and production.
- Provide high-level guidance to engineering squads in India and the US, defining schemas and governance models for BigQuery and Databricks.
- Work with engineers across multiple geographies and time zones.
View Full Description & ApplyYou'll be redirected to the employer's site
