Senior Data Engineer

5+ years of professional experience in data engineering or related fields. Strong programming skills in Python and experience with Spark/PySpark. Deep experience with AWS cloud services for data processing (Glue, S3, Lambda, Step Functions, ECS, etc.). Proficiency in SQL and Bash scripting. Hands-on experience with container technologies like Docker. Knowledge of CI/CD pipelines and version control using Git. Experience working with Snowflake or similar cloud-based data warehouses. Solid understanding of data integration, ETL best practices, and large-scale data processing.

Design, build, and maintain scalable and reliable data pipelines. Extract, transform, and load large datasets from various structured and unstructured data sources using Spark/PySpark. Develop and integrate data processing solutions leveraging AWS services such as Glue, S3, Step Functions, Lambda, EC2, Batch, ECR, and ECS. Write and optimize complex SQL queries for large-scale data processing and analysis. Translate business and technical requirements into detailed, functional data solutions. Collaborate with cross-functional teams to support data initiatives and ensure consistent data delivery. Implement best practices for code versioning (Git), CI/CD processes, and containerization (Docker). Utilize Snowflake for scalable data storage and analytics.