Senior Data Engineer (Databricks)
New
C
Chamberlain GroupSmart Home / IoT
Bulgaria, Georgia, Lithuania, Mexico, Moldova, Poland, UkraineFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English (B2 level or higher)
- Experience
- 5+ years
- Required Skills
- PythonSQLETLData engineeringRESTful APIsDatabricksPySpark
Requirements
- 5+ years of Data Engineering experience.
- Strong experience with Databricks.
- Expert-level Python and PySpark proficiency.
- Strong SQL skills.
- Experience designing and implementing Medallion Architecture (Bronze, Silver, Gold).
- Experience building scalable ETL/ELT data pipelines.
- Experience integrating data from REST APIs and third-party services.
- Experience developing real-time or near real-time data pipelines.
- Experience optimizing Spark workloads for performance and scalability.
- Understanding of data quality, governance, and monitoring best practices.
- Bachelor's degree in Computer Science or related field.
- English proficiency at B2 level or higher.
Responsibilities
- Design, build, and maintain scalable data pipelines in Databricks.
- Develop API integrations to ingest data from multiple internal and external systems.
- Implement and maintain a Medallion Architecture (Bronze, Silver, Gold) to support reliable and scalable analytics.
- Build and optimize batch and real-time data processing pipelines with a focus on low latency and high reliability.
- Develop reusable data transformation frameworks using Python and PySpark.
- Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
- Ensure data quality through validation, monitoring, and automated testing.
- Collaborate with architects, product owners, and engineering teams to design scalable data solutions.
- Troubleshoot production issues and improve pipeline reliability.
- Document data flows, architecture decisions, and operational procedures.
View Full Description & ApplyYou'll be redirected to the employer's site
