- Design, build, and maintain scalable data pipelines and platform infrastructure on Snowflake.
- Architect end-to-end data flows from source ingestion through silver and gold layers.
- Maintain platform reliability, performance, and ongoing operations; evaluate and refactor systems for scale, cost efficiency, and maintainability.
- Establish version control, CI/CD, testing, and documentation standards for platform components.
- Design and maintain Apache Airflow or equivalent orchestration workflows, including monitoring, alerting, logging, and automated retries.
- Build and maintain tested, documented, version-controlled gold-layer models and a governed metrics layer in Snowflake using dbt.
- Define reusable metric definitions and push transformation logic upstream into the warehouse.
- Partner with analytics and BI stakeholders on the boundary between data engineering and analytics or reporting.
- Structure and document data models to support governance and consistency for AI agents and autonomous analytical systems.
- Use coding agents and AI-assisted tooling to develop pipelines, models, documentation, and tests.
SQLApache AirflowGit+5 more