Sr. Data Engineer
New
A
AllocateFintech
Fully Remote Position (All I-9 eligible candidates will be considered)Full-TimeSenior
Salary$145K - $220K
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- AWSDockerPythonSQLETLKubernetesMachine LearningData engineeringCI/CDData modeling
Requirements
- 5+ years of hands-on experience in data engineering or related fields.
- Strong experience with AWS cloud services (S3, EC2, ECS, EKS, Athena, Redshift, Glue, Step Functions).
- Proficiency in SQL, relational database design, and query optimization.
- Experience with data warehouses or lakehouses (e.g., Snowflake, Databricks).
- Fluency in at least one major programming language such as Python; familiarity with TypeScript/Node.js or C# is a plus.
- Experience with ML workflows, including training datasets and feature stores.
- Understanding of vector databases (pgvector, Chroma, Pinecone) and AI frameworks (LangChain, LlamaIndex).
- Experience with containerization (Docker) and Kubernetes (AWS EKS).
- Familiarity with CI/CD, data orchestration, and workflow managers (Airflow, Prefect, dbt).
- Excellent communication and collaboration skills in a remote environment.
- Bachelor’s degree in Computer Science or related field, or equivalent experience.
Responsibilities
- Build and extend the data lakehouse on AWS combining storage and warehouse technologies.
- Contribute to knowledge graph modeling and vector database integration for AI agents.
- Develop robust ETL/ELT pipelines for batch and real-time data ingestion and transformation.
- Prepare training datasets, feature stores, and workflows to support ML model and AI feature deployment.
- Collaborate with DevOps to deploy, containerize, and orchestrate data services using Docker/Kubernetes on AWS EKS.
- Implement CI/CD pipelines for automated testing and deployment of data workflows.
- Monitor performance, troubleshoot production issues, and recommend new technologies to improve pipeline reliability.
View Full Description & ApplyYou'll be redirected to the employer's site
