Sr Data Engineer
New
P
ProtectiveInformation Technology
Birmingham, AL / Nebraska - Virtual / Work From HomeFull-TimeSenior
Salary$84,000 - $128,667 a year
Apply NowOpens the employer's application page
Job Details
- Experience
- 3+ years
- Required Skills
- PythonSQLSparkdbtDatabricksAzure DevOps
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field, or equivalent practical experience.
- 3+ years building and supporting production data pipelines in a cloud data platform environment.
- Strong hands-on Python and SQL skills.
- Hands-on experience with Databricks or a comparable Spark-based lakehouse, including Delta Lake tables, MERGE, and incremental load patterns.
- Practical understanding of medallion/multi-layer lakehouse design.
- Experience ingesting data from APIs, relational databases, files, or SaaS applications.
- Working knowledge of dimensional modeling and ELT design patterns.
- Experience with orchestration and scheduling tools like Dagster, Airflow, or Azure Data Factory.
- Proficiency with Git-based source control, pull request review, and CI/CD practices in Azure DevOps or similar.
- Experience troubleshooting production data failures, performance bottlenecks, and source-system changes.
- Familiarity with pipeline monitoring, alerting, and data quality SLAs.
Responsibilities
- Design, develop, and maintain production data pipelines on Databricks using Python, SQL, Apache Spark, and Delta Lake.
- Build Bronze-layer ingestion from various sources using dlt and native ingestion methods.
- Develop Silver-layer transformations in dbt and Python to cleanse and standardize data.
- Create Gold-layer data products including dimensional models and fact/bridge tables.
- Author and maintain data contracts using the Open Data Contract Standard (ODCS).
- Implement data quality checks and orchestrate workflows using Dagster.
- Apply governance via Unity Catalog and manage credentials via Azure Key Vault.
- Troubleshoot production failures and participate in on-call rotations.
- Build and maintain CI/CD pipelines in Azure DevOps.
- Collaborate with data architects, analysts, and business stakeholders.
View Full Description & ApplyYou'll be redirected to the employer's site
