Data Developer (Azure/Databricks) Senior
New
C
CI&TTechnology Services
BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLCI/CDDatabricksGitHubPySpark
Requirements
- Solid experience in Azure Data Services and Databricks.
- Experience with Databricks and PySpark.
- Experience with Medallion Architecture.
- Proficiency in Python and SQL for data manipulation and transformation.
- Knowledge of CI/CD and versioning with GitHub.
- Knowledge of analytical data modeling (ingestion, preparation, and consumption layers).
- Experience with cloud migration projects.
- Experience with ETL/ELT and integration of large data volumes.
- Experience with AI model deployment and tools like Azure ML.
- Experience with AI projects and pipeline automation.
Responsibilities
- Develop and maintain data pipelines (ETL/ELT) in an Azure environment using Databricks for processing and integration.
- Maintain a 360-degree view to understand the project as a whole and the impact of specific solutions.
- Implement CI/CD practices for deployment automation and code versioning in GitHub.
- Model and evolve tables in Unity Catalog, applying naming and schema versioning best practices.
- Design and optimize data models to ensure scalability, performance, and governance.
- Support cloud migration projects ensuring data security, quality, and integrity.
- Collaborate with business and data science teams to deploy AI models in a cloud environment.
- Ensure data quality and consistency using data profiling and monitoring techniques.
- Document data engineering processes, architectures, and best practices.
View Full Description & ApplyYou'll be redirected to the employer's site
