Senior Data Engineer
New
5
540Federal Health IT
Remote within the continental United StatesFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 10+ years
- Required Skills
- PythonSQLETLGitSparkData modelingDatabricks
Requirements
- 10+ years of data engineering, software engineering, or related technical experience
- Extensive hands-on experience designing, building, and operating production data pipelines and data products
- Advanced proficiency with Python and SQL
- Strong experience with Apache Spark and distributed data processing
- Experience working with Databricks or similar modern data platforms
- Experience designing and maintaining ETL/ELT processes for complex, large-scale datasets
- Experience integrating data across disparate systems and developing API-based integrations
- Strong understanding of data modeling, data architecture, data quality, and data governance principles
- Experience troubleshooting and optimizing complex production data pipelines
- Experience working with Git-based development workflows and modern software engineering practices
- Demonstrated experience providing technical guidance and mentoring engineers
- Strong client and stakeholder communication skills
Responsibilities
- Design, develop, and maintain scalable, production-ready data pipelines and data products using Spark (Python/SQL) in a Databricks environment
- Lead the integration and transformation of complex data from diverse DoW and federal health systems into reliable, reusable data products
- Design scalable approaches for data ingestion, integration, and exchange, including API-based integrations and services
- Establish and promote reusable data engineering patterns, standards, and best practices
- Provide technical guidance on data architecture, pipeline design, data modeling, and integration approaches
- Monitor, troubleshoot, and optimize production workflows and data pipelines for performance and reliability
- Define and implement data validation, quality, and governance practices
- Collaborate with engineers, architects, analysts, and customer stakeholders to translate complex data needs into solutions
- Mentor other engineers and drive the adoption of improved engineering tools and processes
View Full Description & ApplyYou'll be redirected to the employer's site
