Software Engineer II (Data Platform)
IndiaFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 3+ years of experience
- Required Skills
- PythonSQLSparkCI/CDTerraformDatabricks
Requirements
- 3+ years of experience in software engineering, with a track record of building production-grade backend or data systems.
- Strong hands-on experience with Apache Spark and the Databricks/Delta Lake ecosystem.
- Expert-level Python and SQL skills.
- Deep understanding of software design principles, debugging, and distributed computing concepts.
- Experience with CI/CD practices.
- Working knowledge of Terraform or similar Infrastructure as Code tools.
- Background in building monitored, observable systems where data quality and validation are treated as first-class citizens.
Responsibilities
- Design and develop core components of the data platform, including high-volume event ingestion pipelines and real-time processing workflows using Apache Spark.
- Develop reusable data access layers and framework-based ETL/ELT processes that transform structured business data and event streams into trusted Lakehouse assets.
- Contribute to the development of internal APIs and backend services that enable other engineering teams to interact with the data platform.
- Build robust, fault-tolerant pipelines with a focus on data correctness, consistency, and availability.
- Implement schema enforcement and validation gates to maintain strict data governance across systems.
- Build and own the observability framework including logging, metrics, and alerting to monitor pipeline health, data freshness SLAs, and system uptime.
- Apply rigorous automated testing, including unit, integration, and regression, to all data code.
View Full Description & ApplyYou'll be redirected to the employer's site
