- Implement new ingestion sources end-to-end including connector code, DAG, schema, monitoring, and catalog registration.
- Investigate and resolve pipeline failures independently for known classes of issues.
- Identify downstream dependencies and potential risks to data quality before writing code.
- Produce clear, structured handovers for escalated issues.
- Review peers' pipeline pull requests to identify technical issues like race conditions or missing idempotency checks.
- Perform structured investigations into data quality issues by tracing lineage.
- Support data security, governance tasks, and reverse ETL integrations.
- Mentor junior data engineers on significant projects and team conventions.
PythonSQLApache Airflow+3 more