Senior Databricks Engineer
New
C
Cronos Europa PortugalTechnology Services
Lisboa, PT; Ability to work effectively in a fully remote environment.Full-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Fluent English, written and spoken, is mandatory.
- Experience
- Minimum 6 years of professional experience in Data Engineering.
- Required Skills
- PythonSQLGitSparkCI/CDDatabricksPySpark
Requirements
- Minimum 6 years of professional experience in Data Engineering.
- Significant hands-on experience with Databricks in production environments.
- Strong expertise in Apache Spark, PySpark, and Spark SQL.
- Proven practical experience with Delta Lake.
- Advanced Python and SQL skills.
- Experience designing, deploying, and supporting production-grade data pipelines.
- Experience with Git and CI/CD practices.
- Knowledge of Databricks Workflows, Jobs, and Delta Live Tables.
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- Fluent English, written and spoken, is mandatory.
Responsibilities
- Design, build, and maintain scalable data pipelines on the Databricks Lakehouse Platform.
- Develop high-quality solutions using PySpark, Spark SQL, and Python.
- Design and optimize Delta Lake tables, including partitioning, schema evolution, and performance tuning.
- Implement layered data architectures (Raw, Cleansed, Curated).
- Build and manage workflows using Databricks Jobs, Workflows, and Delta Live Tables.
- Optimize Spark workloads, including partitioning, caching, shuffling, join strategies, and file sizing.
- Manage cluster configurations and monitor platform cost efficiency.
- Implement logging, monitoring, alerting, and recovery mechanisms to ensure reliable operations.
- Integrate data quality controls throughout the pipeline lifecycle.
- Work within Unity Catalog to support data governance, lineage, and access control.
View Full Description & ApplyYou'll be redirected to the employer's site
