- Build and maintain Microsoft Fabric data pipelines for REST APIs, SQL Server sources, and file-based ingestion.
- Develop and orchestrate Spark notebooks within Microsoft Fabric.
- Implement Bronze, Silver, and Gold lakehouse patterns in OneLake environments.
- Develop ingestion processes that accommodate schema drift and evolving source systems.
- Design reusable ingestion frameworks and scalable data engineering patterns.
- Build, test, and optimize ETL/ELT processes for performance, scalability, and maintainability.
- Support semantic models, Direct Lake solutions, and downstream analytics requirements.
- Participate in Azure DevOps CI/CD processes, including Dev/Test/Prod promotion workflows.
- Implement metadata, lineage, governance, and security practices for Microsoft Fabric solutions.
- Collaborate with client engineers through paired development and knowledge-sharing.
SQLSparkRESTful APIs+1 more