Engenheiro de Dados Sênior GCP/DBT
J
JobgetherFinancial services
Availability to work remotely in Brazil.ContractSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- Minimum 3 years of proven experience with GCP, 3 years of proven experience with dbt, and 3 years of proven experience with PySpark.
- Required Skills
- PythonSQLGCPTerraformBigQuerydbtPySpark
Requirements
- Have at least 3 years of proven experience with GCP.
- Have at least 3 years of proven experience with dbt.
- Have at least 3 years of proven experience with PySpark and strong Python knowledge.
- Demonstrate strong BigQuery expertise, including data modeling, query optimization, partitioning, clustering, data loading, security, and governance.
- Have hands-on Cloud Storage experience, including bucket management, storage classes, lifecycle policies, IAM, and data security.
- Have experience provisioning, configuring, managing, and optimizing Dataproc Spark/Hadoop clusters.
- Know Dataflow, Composer, and dbt for data processing and ELT/ETL orchestration.
- Have advanced SQL skills for BigQuery, dbt, and complex data transformations.
- Know Shell scripting and GitFlow, and have experience with Git, GitHub, or Bitbucket.
- Understand Cloud IAM, VPC, networking, subnets, firewall rules, and cloud security best practices.
- Be familiar with Agile methodologies, ceremonies, and Jira.
- Be available to work remotely in Brazil.
Responsibilities
- Analyze data warehouse architectures and map data, transformations, and processes across GCP services.
- Define data migration strategies and architecture plans for GCP-based data environments.
- Design and optimize BigQuery data models, schemas, partitioning, clustering, and data ingestion.
- Structure Cloud Storage data zones and develop ELT/ETL pipelines with Dataproc, Spark, Dataflow, and dbt.
- Translate business logic and existing transformations into cloud-based solutions and implement data validation and quality mechanisms.
- Provision and manage GCP infrastructure using Terraform, including datasets, tables, buckets, and Dataproc clusters.
- Monitor and optimize BigQuery queries, Spark jobs, and GCP resource consumption.
- Implement data security and governance practices, including IAM policies and data protection.
- Troubleshoot issues and document architectures, data models, pipelines, and operating procedures.
- Participate in Agile workflows and Jira, and contribute to the data community of practice.
View Full Description & ApplyYou'll be redirected to the employer's site
