Senior Data Engineer
New
K
Kratos GrowthConsumer Intelligence
Remote, New York, Country code: US, 4+ hour overlap with U.S. Eastern Time Zone desiredFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years building and maintaining production ETL/ELT data pipelines; 2+ years working with text/NLP data; 3+ years of data products shipped to production.
- Required Skills
- PythonSQLETLKubernetesAzureDatabricksNLPLLMPySpark
Requirements
- 5+ years building and maintaining production ETL/ELT data pipelines
- 4+ years of Python experience in production environments
- 3+ years of SQL experience including complex queries and performance optimization
- 2+ years of experience working with text/NLP data including tokenization and embeddings
- 2+ years of PySpark experience
- 3+ years of data products shipped to production
- 1+ years of Databricks production use including notebooks, Delta Lake, and job scheduling
- 2+ years of cloud platform experience with Azure or equivalent
- Experience deploying containerized applications to Kubernetes
- Bachelor's degree in Computer Science, Data Science, Engineering, or related quantitative field
Responsibilities
- Design, build, and maintain production data pipelines processing 10M+ text records daily across multiple languages
- Architect scalable NLP data infrastructure using PySpark, Databricks, and Azure services
- Integrate Large Language Model APIs into production pipelines for text analysis and enrichment
- Establish DataOps standards including CI/CD, testing frameworks, and deployment automation
- Implement observability and alerting for pipeline health, data quality, and system performance
- Collaborate with data scientists to productionize ML models and NLP systems
- Define data governance frameworks and quality SLAs for enterprise client delivery
- Mentor team members and contribute to technical hiring as the team scales
View Full Description & ApplyYou'll be redirected to the employer's site
