Senior Databricks Platform Engineer
New
J
JobgetherData Engineering
Based in United StatesFull-TimeSenior
Salary169,604 - 229,464 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 10+ years of specialized experience preferred; equivalent experience accepted in lieu of degree (e.g., 14+ years with bachelor's degree)
- Required Skills
- AWSPythonSQLSparkCI/CDDevOpsTerraformDatabricks
Requirements
- Master’s degree in information systems or related field with 12+ years experience (equivalent experience accepted in lieu of degree).
- Strong hands-on experience administering Databricks Lakehouse, Unity Catalog, and cluster management.
- Demonstrated expertise in Databricks performance engineering, Spark optimization, and SQL tuning.
- Proficiency in Python, Bash, and SQL for automation and data engineering.
- Experience operating Databricks in AWS-based or hybrid cloud environments.
- Hands-on experience with CI/CD and DevOps tooling (e.g., Jenkins, GitLab CI, GitHub Actions).
- Familiarity with observability and monitoring tools like Grafana, Prometheus, or Datadog.
- Strong knowledge of cloud security, IAM, networking, and encryption within Databricks.
- Familiarity with federal security and compliance processes (e.g., ATO, FedRAMP) is highly valuable.
- Strong analytical, troubleshooting, and communication skills.
Responsibilities
- Design, configure, and maintain the enterprise Databricks Lakehouse Platform including workspaces, Unity Catalog, and serverless capabilities.
- Develop and optimize ETL/ELT pipelines using Apache Spark, Delta Lake, Delta Live Tables, and Databricks Workflows.
- Govern Unity Catalog structures and implement RBAC and access-control policies.
- Automate platform provisioning and deployments using Terraform, Python, SQL, and Databricks Asset Bundles.
- Monitor platform performance and health using observability tools and KPI dashboards.
- Perform Spark and SQL performance tuning and workload optimization.
- Implement cost-management strategies including autoscaling, right-sizing, and DBU consumption analysis.
- Collaborate with security and architecture teams to maintain high-availability, disaster-recovery, and compliance controls.
View Full Description & ApplyYou'll be redirected to the employer's site
