Senior Data Engineer
New
BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLGCPAzureSparkDatabricksGenerative AIPySpark
Requirements
- Strong experience as a Senior Data Engineer.
- Advanced proficiency in Python and SQL.
- Hands-on experience with Databricks, Spark, and PySpark.
- Knowledge of Generative AI concepts (LLMs, RAG, agent-based systems).
- Experience with Azure Databricks and/or Google Cloud Platform.
- Familiarity with real-time data processing using Kafka and event-driven architectures.
- Strong understanding of MLOps practices, CI/CD pipelines, and production deployment.
- Experience in data governance, data quality, and MDM concepts.
- Strong analytical thinking and problem-solving skills.
- Experience working in agile environments.
Responsibilities
- Develop, optimize, and maintain scalable data pipelines using Databricks, Spark, and PySpark.
- Design and implement data solutions that support GenAI use cases, including LLMs and RAG architectures.
- Build and evolve data products in cloud environments (Azure and GCP).
- Enable Master Data Management (MDM) initiatives, including Golden Record creation and data quality improvement.
- Support ML/AI initiatives by preparing and structuring data for machine learning models (MLOps).
- Implement CI/CD pipelines and development best practices using GitHub.
- Collaborate with cross-functional teams to translate complex requirements into scalable data solutions.
View Full Description & ApplyYou'll be redirected to the employer's site
