Senior Platform Data Engineer II

View full description

Requirements:

• Hands-on experience deploying production-quality code in fast-paced environments
• Proficiency in Python (preferred), Java, or Scala for data processing and pipeline development
• Ability to thrive in fast-changing, ambiguous situations, balancing immediate needs with long-term goals
• Experience with data pipeline tools, such as Airbyte for ingestion and dbt for transformation/modeling
• Hands-on expertise with container orchestration tools, such as Kubernetes, and cloud-native environments (e.g., AWS)
• Proficiency with workflow automation and orchestration tools, like Dagster or Apache Airflow
• Deep familiarity with PostgreSQL, including administration, tuning, and provisioning in cloud platforms (e.g., AWS)
• Strong experience in ETL/ELT pipelines and data modeling, including raw vs. curated datasets, star schemas, and incremental loads
• Advanced SQL skills, with expertise in relational databases and data warehouses (especially Snowflake)
• Knowledge of best practices in data governance and security
• Excellent problem-solving skills and ability to troubleshoot complex issues
• Strong communication skills to collaborate with cross-functional teams

Responsibilities:

• Own and optimize data pipeline infrastructure to ensure reliable, efficient, and scalable data flows from diverse sources.
• Contribute to the development of the data engineering roadmap in collaboration with Platform leadership and cross-functional stakeholders.
• Design, build, and maintain scalable ETL/ELT pipelines to transform raw data into curated datasets within AWS S3 and Snowflake.
• Implement and standardize data governance practices, ensuring data quality, lineage tracking, schema consistency, and compliance across pipelines.
• Collaborate with analytics and engineering teams to manage backfills, resolve schema drift, and implement best practices for incremental loads.
• Lead the design and implementation of a layered data architecture to improve scalability, governance, and self-service analytics.
• Develop and implement data contracts by collaborating across teams to align business goals with technical needs.
• Evaluate, plan, and execute new data tools, infrastructure, and system expansions to support company growth and evolving analytics needs.
• Deliver scalable, efficient, and maintainable code by applying architectural best practices and adhering to data engineering standards.
• Maintain SLAs for data freshness, accuracy, and availability by defining clear metrics that foster stakeholder trust and ensure consistent, reliable data delivery.
• Collaborate with the Data Analytics team to facilitate the delivery of strategic initiatives.