Senior Data Engineer
New
F
Fox Point Recruitment LLCData Engineering
IndiaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of professional experience in data engineering or related roles; 3+ years of hands-on experience with cloud-based data platforms
- Required Skills
- SQLCloud ComputingETLData engineeringSparkDatabricksPySpark
Requirements
- 5+ years of professional experience in data engineering or related roles
- 3+ years of hands-on experience with cloud-based data platforms
- Proficiency with Azure Data Factory (ADF) for data integration solutions
- Hands-on experience with Databricks platform and workspace management
- Strong expertise in distributed data processing using Apache Spark framework
- Experience building declarative, maintainable data pipelines
- Advanced proficiency in writing optimized SQL queries for complex data transformations
- Strong programming skills in PySpark for data processing and ETL operations
- Bachelor's Degree in Computer Science, Engineering, Mathematics or related field OR equivalent professional experience
Responsibilities
- Design and develop robust, scalable data pipelines using Azure Data Factory (ADF)
- Build and optimize data processing workflows using Databricks and Apache Spark
- Implement declarative pipeline solutions using Lakehouse architecture and best practices
- Write and optimize complex SQL queries for data transformation and analysis
- Develop PySpark applications for large-scale data processing
- Collaborate with data scientists, analysts, and stakeholders to understand data requirements
- Ensure data quality, security, and compliance across all pipelines
- Monitor, troubleshoot, and optimize data pipeline performance
- Document technical solutions and maintain best practices documentation
- Mentor junior team members and contribute to team knowledge sharing
View Full Description & ApplyYou'll be redirected to the employer's site
