Data Engineer
New
I
InductiveHealthPublic health software
Listing location: United States; Workplace type: RemoteFull-TimeMiddle
Salary130,000 - 145,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 3+ years of experience implementing enterprise-grade data pipelines, or 7+ years of related professional experience (e.g., software, data, or integration engineering) that includes building and operating production data pipelines.
- Required Skills
- AWSPythonSQLMicrosoft AzureApache KafkaGitHub Actions
Requirements
- Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field.
- 3+ years implementing enterprise-grade data pipelines, or 7+ years of related professional experience that includes building and operating production data pipelines.
- Hands-on expertise with NiFi, Kafka, Python, and SQL, including cloud-based ETL/ELT pipeline design, operation, and optimization.
- Experience building production-grade pipelines from the ground up and working with distributed data systems.
- Experience deploying and operating pipelines in Azure or AWS.
- Experience securely handling sensitive data, including IAM/RBAC, encryption, and secrets management.
- Experience implementing data-pipeline CI/CD with GitHub and GitHub Actions, including NiFi Registry, automated testing, and peer-reviewed releases.
- Experience using or integrating with OpenSearch or Elasticsearch.
- Practical experience with relational databases and existing schemas.
- Strong understanding of data security and compliance requirements for sensitive data.
- Strong technical documentation skills and ability to collaborate across functions.
Responsibilities
- Design and operate secure, high-performance pipelines for real-time and batch public health data flows.
- Build and optimize ETL/ELT pipelines using NiFi, Kafka, Python, and SQL.
- Build and maintain CI/CD pipelines for NiFi flows, Python, and SQL using GitHub and GitHub Actions.
- Version NiFi flows through NiFi Registry and automate promotion across development, test, and production.
- Deploy and operate data pipelines on Azure and AWS in partnership with DevOps/SRE.
- Apply CI/CD practices including pull request reviews, branch protection, automated testing, environment configuration, and versioned releases with rollback.
- Collaborate with interoperability, product, and data science teams to migrate legacy pipelines to modern architectures.
- Define monitoring and alerting strategies, create runbooks, and troubleshoot production issues.
View Full Description & ApplyYou'll be redirected to the employer's site
