Senior Data Engineer – Platform, Automation & AI Acceleration (GCP)
New
J
JobgetherData engineering
This is a fully remote opportunity ... based in Brazil.Full-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLApache AirflowCI/CDTerraformBigQuery
Requirements
- Have solid data engineering experience on Google Cloud Platform, particularly with BigQuery, Cloud Composer/Airflow, Dataflow, Cloud Run, Cloud Functions, and Cloud Workflows.
- Have advanced proficiency in Python and SQL, including automation, internal tooling, APIs, and GCP SDKs.
- Have experience implementing CI/CD pipelines using GitHub Actions, GitLab CI, or Cloud Build.
- Have practical experience with Terraform, Infrastructure as Code, and DataOps.
- Know SQL analysis and validation tools such as SQLFluff and SQLGlot.
- Have experience developing and maintaining automated pipelines, tests, schema validation, and data governance mechanisms.
- Have hands-on experience with AI-assisted development tools, particularly Antigravity, for code generation and validation.
- Nice to have: experience with Dataform or dbt, including unit testing, ephemeral environments, and Slim CI.
- Nice to have: knowledge of SQL Abstract Syntax Trees, data lineage, and automated anti-pattern detection.
- Nice to have: experience developing AI agents, Model Context Protocol servers, IDE extensions, or automated Pull Request code review bots.
- Nice to have: knowledge of BigQuery cost and performance optimization, including BigQuery Editions and slot consumption.
- Nice to have: experience with Golden Paths, Golden Modules, and Platform Engineering.
Responsibilities
- Develop AI-assisted workflows with Antigravity, including context libraries, templates, tools, and validation mechanisms.
- Build and maintain CI/CD pipelines with automated SQL validation, contract testing, schema checks, data quality controls, and performance testing.
- Implement BigQuery cost analysis and query optimization using tools such as Dry Run and Query Explain.
- Design Terraform infrastructure-as-code solutions and reusable modules for GCP resources, permissions, and governance policies.
- Improve orchestration using Cloud Composer/Airflow, Cloud Workflows, Eventarc, and Pub/Sub.
- Integrate data engineering, automation, and AI tools into development workflows, including automated Pull Request validations.
- Improve Developer Experience by reducing manual tasks, rework, and delivery-cycle time.
- Promote engineering practices supporting platform reliability, scalability, governance, and cost optimization.
View Full Description & ApplyYou'll be redirected to the employer's site