Principal Data Engineer - Remote US

View full description

Requirements:

• Bachelor's degree in Computer Science, Information Systems, or related fields.
• 7+ years of experience as a Data Engineer focusing on ETL processes.
• Professional experience with Spark and AWS pipeline development.
• Strong proficiency in Python and experience with related libraries (e.g., pandas, NumPy, PySpark).
• Hands-on experience with AWS Glue or similar ETL tools.
• Solid understanding of data modeling, warehousing, and architecture principles.
• Expertise in working with large data sets and distributed computing frameworks.
• Experience developing and training machine learning models.
• Strong proficiency in SQL and familiarity with data matching methodologies.
• Experience with data governance and security practices.
• Strong problem-solving and communication skills.

Responsibilities:

• Design, develop, and maintain robust and scalable ETL pipelines to acquire, transform, and load data from various sources.
• Collaborate with cross-functional teams to understand data requirements and develop data acquisition strategies.
• Implement data transformation logic using programming languages like Python.
• Utilize AWS Glue or similar tools to manage ETL jobs and workflows.
• Optimize and tune ETL processes for performance with large datasets.
• Apply data matching and aggregation methodologies to ensure data accuracy.
• Implement data governance practices for compliance and security.
• Collaborate with the data engineering team to adopt new technologies.