- Architect and Scale Data Pipelines: Design, build, and maintain robust ETL/ELT pipelines (both batch and real-time streaming) using modern data orchestration tools (e.g., Airflow, Dagster, Prefect).
- Empower ML and Analytics: Partner closely with Data Science and ML teams to streamline data synchronization, feature engineering, and operationalize ML models.
- Drive Automation and Tooling: Optimize our data infrastructure by automating data flows and enhancing internal analytics pipelines, leveraging modern transformation tools (like dbt) to improve overall analytics efficiency.
- Innovate with Synthetic Data: Develop and implement synthetic data generation pipelines to support robust model training and privacy-safe testing environments.
- Act as a Strategic Partner: Gain deep domain expertise to help shape our data architecture and collaborate with Data/ML teams to define high-impact projects.
- Champion Engineering Excellence: Actively participate in Agile ceremonies (sprint planning, retrospectives) and drive best practices in code reviews, CI/CD, and data quality testing.
- Ensure Data Governance: Design systems that strictly adhere to company security, privacy, and compliance policies (e.g., role-based access control, data retention etc. ).
PostgreSQLPythonSQL+4 more