Data Engineer – Databricks
A
AspenView Technology PartnersHealthcare Technology
Open to mid-senior and senior candidates across Latin AmericaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Advanced/Fluent English (C1/C2)
- Experience
- 4+ years of production experience in data engineering, with at least 2+ years of hands-on expertise building scalable solutions on Databricks.
- Required Skills
- PythonSQLETLDatabricksPySpark
Requirements
- 4+ years of production experience in data engineering.
- 2+ years of hands-on expertise building scalable solutions on Databricks.
- Strong mastery of PySpark, Delta Lake, Databricks SQL, and Unity Catalog.
- Advanced proficiency in Python and SQL.
- Hands-on experience with managed cloud data storage and computing services (AWS S3/EMR or Azure ADLS Gen2/Data Factory).
- Experience with Git, CI/CD deployment pipelines, and data validation frameworks.
- Advanced/Fluent English (C1/C2) for daily active collaboration.
- Familiarity with healthcare data standards (FHIR, HL7, OMOP, ICD-10/CPT codes) or EHR data structures (Epic/Cerner) is preferred.
- Analytical rigor and clear technical communication skills.
Responsibilities
- Design, develop, and maintain automated ETL/ELT data pipelines on Databricks utilizing PySpark, Delta Lake, and Databricks Workflows or Delta Live Tables (DLT).
- Implement Medallion Architecture principles to clean, transform, and aggregate raw clinical/operational datasets into analytics-ready models for BI reporting and predictive analytics.
- Ingest and normalize healthcare data sources, including Epic EHR databases, FHIR/HL7 data feeds, and third-party claims systems.
- Enforce row/column-level access control and data lineage using Databricks Unity Catalog, ensuring compliance with HIPAA, HITRUST, and internal data governance protocols.
- Optimize compute cluster utilization, SQL query performance, Delta table indexing, and memory management.
- Partner with US-based data scientists, business intelligence developers, and clinical stakeholders to deliver reliable data products.
View Full Description & ApplyYou'll be redirected to the employer's site
