Data Engineer (Mid Level)
New
I
Irth SolutionsCloud SaaS
Remote – IndiaFull-TimeMiddle
SalaryCompetitive Salary – A competitive compensation package based on experience and qualifications.
Apply NowOpens the employer's application page
Job Details
- Experience
- 3–5 years
- Required Skills
- SQLETLMicrosoft AzureSparkCI/CDDatabricksPySpark
Requirements
- 3–5 years of experience in Data Engineering, ETL development, or cloud data platform engineering.
- Hands-on experience with Databricks, Apache Spark, PySpark, or distributed data-processing technologies.
- Strong proficiency in SQL, including transformation, joins, aggregations, and performance-aware query development.
- Experience with at least one major cloud platform (Microsoft Azure preferred; AWS/GCP valuable).
- Understanding of data modeling, data quality, schema evolution, and pipeline troubleshooting.
- Knowledge of data-security practices, including RBAC, encryption, and credential/secret management.
- Understanding of Delta Lake, medallion architecture, and lakehouse best practices.
- Experience with metadata and governance platforms like Unity Catalog or Microsoft Purview.
- Familiarity with workflow orchestration tools such as Azure Data Factory, Databricks Workflows, or Apache Airflow.
- Experience with Git-based development, CI/CD, and DevOps practices.
Responsibilities
- Build, maintain, and enhance data ingestion pipelines across AWS, Azure, and GCP.
- Develop both batch and streaming pipelines using Databricks, Spark, and SQL.
- Implement Bronze-Silver-Gold medallion architecture and Change Data Capture (CDC) patterns.
- Configure and maintain Delta Lake storage structures and optimize performance using OPTIMIZE, Z-ORDER, and VACUUM.
- Implement automated data-quality checks, validation, and monitoring.
- Build and maintain production workflows using Databricks Workflows, Delta Live Tables, or Azure Data Factory.
- Contribute to CI/CD pipelines and manage DEV-QA-PROD deployment processes.
- Collaborate with architects and stakeholders to translate design patterns into production-ready solutions.
View Full Description & ApplyYou'll be redirected to the employer's site
