Senior Databricks Data Engineer
New
J
JobgetherData engineering
Based in BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- SQLGitSparkCI/CDDatabricksPySpark
Requirements
- Bring strong hands-on experience with PySpark, Apache Spark, and SQL.
- Have practical experience with Databricks, including Delta Lake, Delta Tables, Unity Catalog, and Lakeflow Declarative Pipelines.
- Have experience with Databricks Jobs, Workflows, orchestration, and pipeline monitoring.
- Understand Lakehouse architecture and the Medallion model across Bronze, Silver, and Gold layers.
- Have experience migrating data platforms or delivering data engineering projects using Databricks.
- Know Git, CI/CD, and modern development and deployment practices.
- Have experience with or knowledge of technologies such as Airflow, Kafka, dbt, AWS Glue, and BigQuery.
- Have experience with SQL and NoSQL databases, such as PostgreSQL, MongoDB, or Cassandra.
- Be able to translate business requirements into scalable data solutions and collaborate with technical and business stakeholders.
- Experience with Lakeflow Connect for data ingestion is desirable.
- Databricks Certified Data Engineer Associate certification is an advantage.
- Experience with accounting or financial projects is a plus.
Responsibilities
- Design, build, migrate, and evolve data engineering solutions within Databricks environments.
- Develop and maintain scalable, reliable, high-performance data pipelines.
- Implement data ingestion, transformation, and delivery processes using Apache Spark and PySpark.
- Design and implement Data Lake and Data Lakehouse architectures using Bronze, Silver, and Gold layers.
- Translate business rules and requirements into efficient data processes and technical solutions.
- Participate in data platform migration and modernization projects.
- Configure and manage Databricks Jobs, Workflows, orchestration, and pipeline monitoring.
- Implement data solutions using Delta Lake, Delta Tables, Unity Catalog, and Lakeflow Declarative Pipelines.
- Ensure pipeline quality, performance, observability, reliability, and operational stability.
- Apply version control, CI/CD, and software engineering best practices throughout development and deployment.
View Full Description & ApplyYou'll be redirected to the employer's site
