Senior Data Engineer
New
C
CI&TFinancial data
Structured job location: Brazil; Workplace type: RemoteFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English proficiency for interaction with global teams, technical discussions, and documentation.
- Required Skills
- PythonSQLDatabricksPySpark
Requirements
- Strong experience with Databricks, including notebooks, jobs, workflows, clusters, and scalable data-solution best practices.
- Proficiency in PySpark, Python, and SQL for transforming raw data into curated datasets.
- Experience with Delta Lake and Lakehouse architecture, including Delta tables, incremental processing, data optimization, and structured data layers.
- Knowledge of the Azure data ecosystem, especially Azure Data Lake Storage, data integration patterns, and cloud-based data processing.
- Experience supporting financial data pipelines, preferably involving reconciliation, P&L, financial closing, or other critical business processes.
- Hands-on experience maintaining and improving pipelines, including troubleshooting, performance tuning, documentation, and production support.
- Ability to work with structured and semi-structured data and ensure data quality, traceability, and readiness for analytical and business use.
- Ability to investigate complex data, understand root causes, and propose practical solutions.
- Interest or experience with AI models and advanced data capabilities.
- English proficiency for interaction with global teams, technical discussions, and documentation.
- Nice to have: familiarity with AI frameworks and methodologies.
- Nice to have: experience with Delta Lake optimization, Databricks Workflows, Databricks Apps, or Streamlit.
- Nice to have: experience with data quality and observability practices, including validation rules, reconciliation checks, monitoring, logs, alerts, and pipeline execution metrics.
Responsibilities
- Develop, maintain, and improve data pipelines using Databricks, PySpark, Python, and SQL.
- Integrate new data sources into the existing data ecosystem.
- Support financial reconciliation, P&L routines, and global closing processes by ensuring data consistency, accuracy, and traceability.
- Process structured and semi-structured data and optimize transformations, storage formats, and query performance using Delta Lake and Lakehouse practices.
- Monitor, troubleshoot, and support existing pipelines, investigating issues and discrepancies in daily and monthly processing.
- Implement improvements to performance, scalability, maintainability, and data quality.
- Collaborate with data engineers, business stakeholders, finance teams, and technology teams to deliver data solutions.
- Support data governance, validation rules, auditability, documentation, and controls.
- Support initiatives involving AI models and integrate AI-driven insights and automation into data workflows.
View Full Description & ApplyYou'll be redirected to the employer's site
