Lead Engineer, Big Data - Databricks
New
J
JobgetherHealthcare Technology
Full-time remote work opportunity within the United States.Full-TimeLead
Salary96,325.57 - 208,705.4 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 6 years of experience in data management or data engineering
- Required Skills
- PythonSQLMicrosoft Power BIAzure DevOpsHIPAAPySpark
Requirements
- At least 6 years of experience in data management or data engineering across cloud platforms.
- Bachelor's degree in IT, Computer Science, Statistics, Business, Healthcare, Engineering, or related field.
- Strong hands-on experience with Azure Databricks, Data Factory, Synapse, Data Lake, and Azure Machine Learning.
- Demonstrated experience implementing AI-assisted development and machine learning solutions.
- Strong proficiency in Python, PySpark, Scala, and SQL.
- Experience designing real-time streaming architectures (e.g., Kafka, Azure Event Hubs, Spark Streaming).
- Experience integrating data platforms with systems such as Pega and Salesforce.
- Solid understanding of HIPAA requirements and healthcare data governance.
- Experience with Azure DevOps CI/CD pipelines and IaC tools like Terraform or ARM templates.
- Experience with Microsoft 365, Power BI, and Azure DevOps.
- Ability to mentor engineers and communicate complex technical concepts to executive audiences.
- Azure Data Engineer certification is preferred.
Responsibilities
- Lead end-to-end data engineering initiatives as a Forward Deployed Engineer.
- Provide technical leadership across Azure Databricks, Azure Synapse, Azure Data Factory, and Azure Data Lake Storage.
- Design and implement scalable, high-performance data architectures for large healthcare datasets.
- Champion AI-first development practices and Azure Machine Learning solutions.
- Develop business intelligence solutions using Power BI and Tableau.
- Build reusable data-quality frameworks using Azure DevOps.
- Lead integration initiatives connecting the Azure data platform with Pega and Salesforce.
- Implement batch and real-time data pipelines supporting claims processing and population health analytics.
- Maintain compliance with HIPAA requirements and healthcare data governance standards.
- Train and mentor team members across Azure data technologies.
View Full Description & ApplyYou'll be redirected to the employer's site
