TY - Data Engineer - 0016

View full description

Requirements:

• Overview of the data life cycle with a focus on data integrity and management.
• Proficiency in Python with experience in Object-Oriented Programming (POO), scripting, and modular coding.
• Strong SQL fundamentals, including subqueries, window functions, and performance optimization.
• Experience in designing and planning databases and data models.
• Knowledge of data warehousing techniques like ETL and dimensional modeling.
• Proficiency with data pipeline management tools, specifically Airflow and Google Cloud dataflow.
• Familiarity with Google Cloud Platform, especially BigQuery.
• Experience in real-time processing using tools like Pub/Sub.
• Understanding of data architecture principles and practices.
• Experience with DevOps practices, including CI/CD, Terraform, and Jenkins.

Responsibilities:

• Design and develop data pipelines for efficient data handling.
• Manage data integration and processing to ensure seamless workflows.
• Implement ETL processes and dimensional modeling for data warehousing.
• Utilize tools like Airflow and Google Cloud dataflow for workflow management.
• Ensure data architecture and security throughout the data lifecycle.