- Design, implement, and maintain scalable data pipelines on the Databricks Lakehouse Platform using technologies such as Apache Spark and Delta Lake.
- Develop and optimize batch and streaming data workflows that support analytics, machine learning, and business intelligence use cases.
- Collaborate with cross-functional teams to translate business and data requirements into reliable, production-ready solutions.
- Improve the performance, scalability, and cost efficiency of data processing workloads within Databricks environments.
- Support the implementation of best practices for data quality, governance, security, and compliance.
- Contribute to architectural discussions and help shape modern data platform strategies for clients.
- Mentor and support junior engineers while fostering knowledge sharing across the team.
AWSSQLGCP+7 more