Senior Data Engineer
New
A
ArionkoderAI consulting
Location: LATAMFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- C1 English
- Experience
- 6+ years of professional data engineering; 3+ years hands-on with Databricks in production
- Required Skills
- PythonSQLData modelingDatabricks
Requirements
- 6+ years of professional data engineering building production-grade pipelines and data models.
- 3+ years of hands-on Databricks experience in production, including Unity Catalog.
- Expert SQL and Python skills for writing, debugging, optimizing, and refactoring pipelines independently.
- Strong data modeling skills, including SCD Type 2, effective dating, temporal validity, provenance, and lineage.
- Experience implementing data quality tests and expectations in pipelines, using dbt tests, Great Expectations, or similar, and CI/CD for data.
- Experience reconciling data across systems through entity resolution or master data practices.
- Experience with incremental and change-driven processing, such as CDC, streaming, or incremental frameworks, against freshness targets.
- Production experience implementing access control, such as row-level security, column masking, or role- or attribute-based access.
- Experience building APIs or services that serve data.
- Daily use of AI coding tools such as Claude, Cursor, or similar, with judgment to review their output.
- C1 English and comfort working fully remotely and directly with a client team.
Responsibilities
- Build and maintain ingestion and ETL pipelines from business systems, operational tools, and document repositories.
- Build Databricks data lakes and choose whether each source should be ingested, indexed and linked, or queried live.
- Implement entity models and resolve identities across systems, escalating ambiguous matches to the designated owner.
- Model facts with source, validity period, validation status, and documented exceptions.
- Turn approved business definitions into certified tables and record owner sign-off and authority rules.
- Implement Databricks access policies and enforce isolation across database, tool, and application layers.
- Integrate structured sources, meet freshness targets, and expose freshness with each fact.
- Own data quality checks, observability, dependency tracking, and cost visibility.
- Build APIs, webhooks, and MCP tools that serve data with caller identity and permissions, and log access with source details.
- Collaborate with the AI engineer, full-stack engineer, data quality lead, and client team; document and hand over the work.
View Full Description & ApplyYou'll be redirected to the employer's site
