Data Engineer
New
I
Irth SolutionsSoftware Product
IndiaFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 3–5 years
- Required Skills
- AWSSQLETLGCPAzureSparkCI/CDDatabricksPySpark
Requirements
- 3–5 years of experience in data engineering, ETL, or cloud data platforms.
- Experience with Databricks, Spark, PySpark, or distributed data processing.
- Strong SQL and structured data transformation skills.
- Experience with at least one cloud platform (Azure preferred; AWS/GCP acceptable).
- Knowledge of data modeling, schema evolution, pipeline troubleshooting, and data quality.
- Understanding of security practices including RBAC, encryption, and credential management.
- Bachelor’s or master’s degree in computer science, Software Engineering, or a related field, or equivalent professional experience.
Responsibilities
- Build and maintain ingestion pipelines across AWS, Azure, and GCP.
- Implement batch and streaming pipelines using Databricks, Spark/PySpark, SQL, Delta Live Tables, and Lakeflow.
- Apply medallion architecture patterns for structured data transformation.
- Implement CDC, SCD Type 1/2, schema evolution, and data validation.
- Manage Delta Lake storage, tables, partitions, and performance optimization.
- Implement data quality validation, profiling, and monitoring aligned with governance standards.
- Build and manage workflows using Databricks Workflows, Delta Live Tables, Azure Data Factory, or similar tools.
- Support CI/CD pipelines, code versioning, and deployment automation.
View Full Description & ApplyYou'll be redirected to the employer's site
