- Evaluate and refine Microsoft Fabric architecture, metadata structures, table-level event triggering, and reusable ingestion patterns.
- Design and implement a metadata-driven ingestion framework for configuration-based onboarding of tables and sources.
- Migrate prioritized data sources and workloads from Azure Synapse into Microsoft Fabric.
- Develop and orchestrate Fabric data pipelines and Spark notebooks using PySpark and Spark SQL.
- Analyze Azure Synapse pipelines and dependencies, and translate existing processing logic into Fabric solutions.
- Build validation and reconciliation processes, including duplicate detection, key alignment, and record-level comparisons.
- Analyze source loading requirements and recommend full-load, incremental, or change data capture patterns.
- Implement Git-based CI/CD and automated environment promotion for Fabric artifacts using GitHub Actions, Azure DevOps, or similar tools.
- Use Claude Code to specify implementations, accelerate development, and review generated code.
- Collaborate with client engineers through paired development, code reviews, and knowledge-sharing sessions.
SQLDatabricksGitHub Actions+1 more