Data Engineer | B2B or CIM

View full description

Requirements:

• Designing, developing, and maintaining scalable data pipelines using Python and SQL.
• Implementing distributed data processing systems with Apache Spark and Apache Airflow.
• Managing ETL processes using Matillion.
• Maintaining cloud-based data platforms using AWS and Azure, including integration with Snowflake and Databricks.
• Designing data quality frameworks using DBT and Great Expectations.
• Excellent understanding of data warehousing concepts.
• Designing scalable data architecture with frameworks like Data Mesh and Data Vault.

Responsibilities:

• Design and build scalable data pipelines and infrastructures.
• Maintain infrastructures that support data-driven decision-making.
• Collaborate with cross-functional teams.
• Work on cloud platforms and distributed data processing frameworks.