Senior Data Engineer
New
C
Capital Technology GroupFederal data engineering
Australia, Canada, Germany, India, United Kingdom, United StatesFull-TimeSenior
Salary130,000 - 165,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 9+ years of professional experience in data engineering or related domains
- Required Skills
- PythonSQLdbtDatabricks
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related technical field.
- 9+ years of professional experience in data engineering or related domains.
- Hands-on experience with Databricks, Apache Spark (PySpark), Python, SQL (PostgreSQL), and dbt.
- Experience designing, building, and maintaining AWS-native data platforms, including AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), Lambda, Step Functions, S3, Redshift, RDS, DMS, and CloudWatch.
- Experience developing data pipelines, workflow orchestration, and data integration solutions.
- Experience with Apache Iceberg and data formats including Parquet, ORC, and Avro.
- Experience designing and optimizing solutions using relational and NoSQL databases, including PostgreSQL, Redshift, Oracle, and GraphDB.
- Experience with performance tuning, system optimization, and enterprise-scale ETL/ELT architectures.
- Java development and modern CI/CD practices using Harness.
- Experience working in Agile, iterative software development environments.
- Applicants must be US Citizens and be able to obtain Public Trust clearance.
Responsibilities
- Design, build, and maintain scalable data pipelines, ETL/ELT workflows, and data models.
- Develop and optimize AWS-native data platforms using AWS data services.
- Build ingestion, transformation, and orchestration workflows for structured and semi-structured data.
- Design and optimize analytical data platforms and integrate enterprise and external data sources.
- Build AI-enabled data solutions using Amazon Bedrock, RAG pipelines, and vector search technologies.
- Improve platform reliability, scalability, performance, and maintainability through monitoring, troubleshooting, automation, and optimization.
- Support analytics and reporting solutions in AWS-based federal data environments that comply with FedRAMP and NIST 800-53 controls.
- Migrate legacy platforms and workflows to cloud-native AWS services.
- Mentor junior engineers through technical guidance, architecture discussions, and code reviews.
- Collaborate with cross-functional teams in an Agile environment to define requirements and deliver data solutions.
View Full Description & ApplyYou'll be redirected to the employer's site
