Grupo QuintoAndar | Data Engineer Specialist

View full description

Requirements:

• Has 7 or more years of experience in Data Engineering roles
• Specialist in technologies, solutions, and concepts of Big Data (Spark, Hadoop, Hive, MapReduce) and multiple languages (YAML, Python)
• Experience with Airflow, Spark, AWS and Databricks
• Strong foundation in software engineering principles, with experience working on data-centric systems
• Experience with columnar storage solutions and/or data lakehouse concepts
• Proficiency in Python, or one of the main programming languages, and a passion for writing clean and maintainable code
• Strong knowledge in optimizing SQL query performance
• Experience in building multidimensional data models (Star and/or Snowflake schema)
• Understanding of the data lifecycle and concepts such as lineage, governance, privacy, retention, anonymization, etc.
• Knowledge in infrastructure areas such as containers and orchestration (Kubernetes, ECS), CI/CD strategies, infrastructure as code (Terraform), observability (Prometheus, Grafana), among others
• Proficiency in English

Responsibilities:

• Build and maintain a high-performance data platform
• Create and edit data pipelines
• Create data modeling and transformation workflows
• Be responsible for the entire code development lifecycle (monitoring deployment, documentation, performance, security, adding metrics and alarms, ensuring SLO budget compliance, and more)
• Investigate inconsistencies and be able to trace the source of differences (data troubleshooting)
• Enable teams across the company to access and use data more effectively through self-service tools and well-modeled datasets
• Align with stakeholders to understand their primary needs, while also having a holistic view of the problem and proposing extensible, scalable, and incremental solutions
• Conduct PoCs and benchmarks to determine the best tool for a given problem, and decide whether to use an off-the-shelf solution or develop one in-house
• Contribute to defining the strategic vision, crossing team and service boundaries to solve problems
• Advocate for the value of data analytics and engineering within the organization and fostering a data-driven culture
• Be a reference within the chapter on technical concepts, tools, and/or best coding practices