Senior Data Engineer
New
N
NexthireSoftware Development
Abu Dhabi , Gurgaon, Jaipur , Indore, INFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5–9 Years
- Required Skills
- PythonSQLETLPandasPySpark
Requirements
- 5–9 years of professional data engineering experience.
- Strong proficiency in Python and SQL.
- Extensive experience with Azure Data Factory (ADF), Azure Databricks, Azure Data Lake, and Azure Synapse Analytics.
- Experience with data transformation using PySpark and Pandas.
- Knowledge of Medallion Architecture and Lakehouse concepts.
- Familiarity with CI/CD pipelines and deployment automation.
- Experience with Microsoft Purview or Informatica (Preferred).
- Exposure to Power BI or reporting/visualization tools (Preferred).
- Experience supporting AI/ML data engineering workflows (Preferred).
- Familiarity with cloud security and governance best practices (Preferred).
Responsibilities
- Design, develop, and maintain scalable end-to-end data pipelines using Python, SQL, and Azure services.
- Build and optimize ETL/ELT workflows for large-scale enterprise data processing.
- Work extensively with Azure Data Factory (ADF), Azure Databricks, Azure Data Lake, and Azure Synapse Analytics.
- Develop scalable data ingestion frameworks for APIs, relational databases, flat files, and cloud data sources.
- Perform data transformation, cleansing, validation, and performance optimization using PySpark and Pandas.
- Implement modern data lake and lakehouse architectures including Medallion Architecture.
- Write optimized SQL queries, stored procedures, and data validation logic.
- Monitor, troubleshoot, and improve pipeline performance and data quality.
- Implement CI/CD pipelines, deployment automation, and version control best practices.
View Full Description & ApplyYou'll be redirected to the employer's site
