Data Solution Engineer
New
A
AretumData engineering
This is a remote positionFull-TimeMiddle
Salary90,000 - 105,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 3 -5 years of data engineering
- Required Skills
- PythonSQLSparkDatabricks
Requirements
- BS/BA degree and 3–5 years of data engineering experience, or an equivalent combination of education and experience.
- Hands-on experience with Databricks, Spark, and YAML.
- Demonstrated experience designing data governance, tagging, and metadata standards.
- Deep understanding of medallion architecture and how metadata, sensitivity, and access requirements evolve across Bronze, Silver, and Gold.
- Experience implementing column masking, row-level security, and dynamic access controls.
- Strong SQL and Spark skills.
- Proficiency in defining pipeline configuration and data contracts declaratively, such as with YAML, as version-controlled, enforceable artifacts.
- Ability to reconcile competing requirements across stakeholders.
- Exposure to a cybersecurity environment is preferred.
- Travel may be required for project needs, client meetings, team collaboration events, or training; expected travel is less than 10%.
Responsibilities
- Develop, test, and implement data pipelines in Databricks using SQL, Python, and Apache Spark.
- Build and maintain pipelines across bronze, silver, and gold layers of the medallion architecture.
- Implement data governance tagging, metadata standards, and classification roles in Unity Catalog.
- Apply data protection policies, including column masking, access controls, and reference data lookups.
- Define and maintain pipeline and workflow configurations using YAML and configuration-as-code practices.
- Execute data quality testing and validation for accuracy, completeness, consistency, and compliance with data contracts and business rules.
View Full Description & ApplyYou'll be redirected to the employer's site
