Senior Data Engineer
New
C
CI&TFinancial data
Listing location: Brazil; Workplace type: Remote; Structured job location: BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLAzureSparkData modelingPySpark
Requirements
- Have solid experience in data engineering and building pipelines in the Azure ecosystem.
- Use advanced SQL for data manipulation, transformation, and analysis.
- Have experience with Python and/or PySpark for ingestion and transformation scripts.
- Have experience with Azure Databricks, including Spark, PySpark, and Spark SQL.
- Have experience with Azure Data Factory (ADF) for pipeline orchestration and automation.
- Have experience with KPIs and indicators in the financial or banking sector.
- Have experience with Bronze, Silver, and Gold layered architecture (Medallion) and Delta Lake.
- Understand analytics data modeling, including Star Schema, OBT, and fact and dimension tables.
- Be familiar with Data Lakehouse environments and MERGE/UPSERT operations.
- Have experience with data quality and governance.
Responsibilities
- Develop scripts and pipelines to load data into the Gold layer, ensuring the quality, consistency, and traceability of consolidated information.
- Structure tables containing platform performance, usage, and availability indicators.
- Create optimized analytical models for consumption by business and management teams.
- Ensure quality, versioning, and documentation standards for pipelines and data models.
- Work with product, operations, and management teams to translate business needs into data engineering solutions.
View Full Description & ApplyYou'll be redirected to the employer's site
