Staff Data Engineer

View full description

Requirements:

• 6+ years in designing, building, and maintaining Data infrastructure and the ability to lead complex projects and teams.
• Bachelor's, Master's or PhD degree in computer science, computer engineering, or a related technical discipline or equivalent industry experience.
• Deep knowledge of Kafka.
• Proficiency in programming languages like Python and Scala.
• Strong knowledge of distributed computing frameworks such as Apache Hadoop and Apache Spark with cloud platforms such as AWS, Azure, or GCP.
• Working experience with Databricks would be nice to have.
• Deep understanding of database design, SQL, and NoSQL databases.
• Experience in managing large datasets and optimizing database performance.
• Proficiency in Git and Terraform and experience in deploying continuous integration and continuous deployment (CI/CD) practices.
• Experience in managing event-driven systems, particularly with Kafka in cloud environments.
• Expertise in developing and implementing data governance frameworks, policies, and procedures to ensure data quality, compliance, and effective data management practices.
• Deep understanding of data security principles, including encryption, decryption, and secure data storage and transfer protocol.

Responsibilities:

• Lead the enhancement of internal processes, focusing on scaling infrastructure, streamlining data delivery, and implementing automation to replace manual operations.
• Design and implement advanced infrastructure for efficient data extraction, transformation, and loading (ETL) using cutting-edge AWS and SQL technologies.
• Develop sophisticated analytical tools that tap into the data pipeline, offering deep insights into crucial business metrics like customer growth and operational efficiency.
• Architect and manage extensive, sophisticated data sets to meet complex business needs and requirements.
• Engage closely with a wide array of stakeholders, including executive, product, data, and design teams, providing high-level support for data infrastructure challenges and advising on technical data issues.
• Collaborate and mentor other senior engineers while providing thoughtful guidance using code, design, and architecture reviews.
• Contribute to defining technical direction, planning the roadmap, escalating issues, and synthesizing feedback to ensure team success.
• Estimate and manage team project timelines and risks.
• Participate in hiring and onboarding for new team members.
• Lead cross-team engineering initiatives.
• Constantly learning about new technologies and industry standards in data engineering.