Mid+/Senior Data Engineer (Python, GCP)

View full description

Requirements:

• Strong cloud provider’s experience on GCP
• Hands-on experience using Python; Scala and Java are nice to have
• Experience in data and cloud technologies such as Hadoop, HIVE, Spark, PySpark, DataProc
• Hands-on experience with schema design using semi-structured and structured data structures
• Experience using messaging technologies – Kafka, Spark Streaming
• Strong experience in SQL
• Understanding of containerisation (Docker, Kubernetes)
• Experience in design, build and maintain CI/CD Pipelines
• Enthusiasm to pick up new technologies as needed

Responsibilities:

• Work alongside clients to interpret requirements and define industry-leading solutions
• Design and develop robust, well tested data pipelines
• Demonstrate and help clients adhere to best practices in engineering and SDLC
• Lead and mentor the team of junior and mid-level engineers
• Contribute to security designs and have advanced knowledge of key security technologies
• Support internal Capco capabilities by sharing insight, experience and credentials