Data Engineer (SC Clearance)

View full description

Requirements:

• Experience with AWS services including S3, Lambda, EMR, SMS, SQS, and additional data infrastructure services.
• Proficiency in Terraform.
• Experience with Databricks and data lake, warehouse, and lakehouse architecture/design.
• Knowledge of Python/Pyspark.
• Familiarity with data platforms and notebooks, including Jupyter, Databricks, and Azure.
• Experience with Gitlab for repository management and CI/CD.
• Java (Spring Boot) experience is a plus.

Responsibilities:

• Design, develop, and maintain scalable and efficient data pipelines for data ingestion, transformation, and loading from various sources.
• Utilize SSIS to create and manage ETL processes for seamless data integration.
• Collaborate with data architects and analysts for implementing efficient data models.
• Ensure data architecture supports business needs and is scalable.
• Demonstrate SSIS expertise for building and managing ETL processes, optimizing performance, and troubleshooting.
• Implement data quality checks within ETL processes, and adhere to governance and compliance standards.