Senior Data Engineer
New
N
NextechHealthcare technology
100% Remote in the United StatesFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 7+ years of production experience in data engineering roles
- Required Skills
- PythonMicrosoft SQL ServerSparkDatabricksPySpark
Requirements
- Have 7+ years of production experience in data engineering, including hands-on experience building and maintaining data pipelines at scale.
- Demonstrate advanced proficiency with Python and PySpark or Apache Spark in production data engineering environments.
- Have hands-on production experience with Databricks, Microsoft Fabric, or comparable modern data platforms.
- Have experience designing, implementing, and operating lakehouse architectures using Delta Lake, Parquet, or equivalent technologies.
- Have experience designing AI-ready schemas, semantic layers, or metadata structures for downstream AI or machine-learning query patterns, or building automated validation for programmatically generated queries.
- Have experience with agentic or automation-driven data quality workflows, such as AI-assisted reconciliation or agent-grounded semantic layers.
- Demonstrate the ability to partner directly with product, analytics, finance, or business stakeholders to shape data requirements proactively.
- Be familiar with Microsoft Azure cloud concepts, including platform as a service, infrastructure as a service, Azure SQL, and hybrid cloud models.
- Be willing and able to develop deeper operational expertise in Azure SQL and SQL Server environments.
- Have meaningful experience mentoring junior or mid-level engineers on technical standards, architecture patterns, and data engineering best practices.
- Preferred: Have production experience with Azure SQL, SQL Server, or Azure SQL Managed Instance, and proficiency with T-SQL and PowerShell.
- Preferred: Have experience with infrastructure-as-code tools such as Terraform, Bicep, or Azure Resource Manager templates.
Responsibilities
- Design and build robust data pipelines and lakehouse solutions using Databricks, Microsoft Fabric, PySpark, and Python.
- Design AI-ready schemas, semantic layers, and metadata structures so AI agents can query data accurately and reliably.
- Build automated validation processes for agent-generated queries.
- Pilot agentic automation for data quality and reconciliation workflows, measuring accuracy and maintaining rollback plans.
- Curate and maintain the semantic layer that grounds AI agents and reduces incorrect responses caused by ambiguous or poorly defined data.
- Catalog and steward enterprise data assets for internal analytics, financial planning, and product engineering.
- Partner with Product Engineering, FP&A, and other stakeholders to identify and understand data requirements.
- Troubleshoot performance issues and support SQL Server and Azure SQL environments.
- Mentor junior engineers on data engineering best practices, architecture patterns, and technical standards.
- Contribute to data protection, security, and compliance reviews, including HIPAA requirements where relevant.
View Full Description & ApplyYou'll be redirected to the employer's site
