Senior Data Engineer
New
M
MachinifyHealthcare intelligence
Remote - USFull-TimeSenior
Salary$180k-$220k
Apply NowOpens the employer's application page
Job Details
- Experience
- 6+ years
- Required Skills
- AWSPythonSQLKafkaAirflowData modeling
Requirements
- 6+ years of experience as a Data Engineer building production-grade pipelines.
- Strong expertise in Python, Spark SQL, and Airflow.
- Experience processing large-scale file-based datasets (CSV, Parquet, JSON) in production environments.
- Experience mapping and standardizing raw external data into canonical models.
- Familiarity with AWS or other cloud platforms, including file storage and distributed compute concepts.
- Experience onboarding new customers and integrating external data with non-standard formats.
- Ability to work across teams and own complex data workflows with minimal supervision.
- Strong written and verbal communication skills.
- Experience working with large-scale or messy datasets such as healthcare, financial, or logs.
- Experience building or willingness to learn streaming pipelines using Kafka or SQS.
Responsibilities
- Design and implement robust, production-grade pipelines using Python, Spark SQL, and Airflow to process high-volume file-based datasets.
- Lead efforts to canonicalize raw healthcare data (837 claims, EHR, partner data, flat files) into internal models.
- Own the full lifecycle of core pipelines from file ingestion to validated, queryable datasets.
- Onboard new customers by integrating their raw data into internal pipelines and canonical models.
- Build resilient, idempotent transformation logic with data quality checks, validation layers, and observability.
- Monitor pipeline health, participate in on-call rotations, and resolve production data flow issues.
- Collaborate with Data Analysts, Data Scientists, Product Managers, and Engineering teams to meet business needs.
- Contribute to the evolution of the data platform including patterns in observability, testing, and automation.
View Full Description & ApplyYou'll be redirected to the employer's site
