Senior Consultant (Databricks)
U
Unison GroupData Engineering
IndiaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- AWSSQLGCPAzureSparkCI/CDDatabricksPySpark
Requirements
- Strong experience with Databricks Platform.
- Hands-on expertise in Apache Spark, PySpark, and Spark SQL.
- Experience with Delta Lake, Unity Catalog, and Databricks Workflows.
- Strong SQL programming and data modeling skills.
- Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform (GCP).
- Experience working with data lakes, data warehouses, and modern data architectures.
- Knowledge of orchestration tools such as Azure Data Factory (ADF), Apache Airflow, or similar.
- Experience with version control tools such as Git.
- Understanding of CI/CD practices and DevOps concepts.
- Excellent analytical, troubleshooting, and problem-solving skills.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks.
- Build and optimize ETL/ELT workflows using PySpark and Spark SQL.
- Develop and manage Delta Lake architecture for reliable and efficient data processing.
- Integrate data from multiple sources including databases, APIs, streaming platforms, and cloud storage.
- Collaborate with business stakeholders, data analysts, and data scientists to understand business requirements.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Implement data quality, governance, and security best practices.
- Monitor production workloads and troubleshoot performance issues.
- Automate deployment processes using CI/CD pipelines.
- Mentor junior team members and provide technical guidance during project delivery.
View Full Description & ApplyYou'll be redirected to the employer's site
