Senior Solution Architect – Spark & Databricks
New
J
JobgetherData engineering
Fully remote working arrangement within the United States.ContractSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- Between 5 and 10+ years of professional experience in data engineering, analytics, or enterprise data solution delivery.
- Required Skills
- SparkDatabricksPySpark
Requirements
- Have 5 to 10+ years of professional experience in data engineering, analytics, or enterprise data solution delivery.
- Have at least 3 years of hands-on Databricks experience, including workload optimization, debugging, performance tuning, and governance.
- Bring expert-level understanding of Apache Spark fundamentals and internals and practical PySpark development experience.
- Have proven Databricks SQL (DBSQL) expertise, including query optimization and performance tuning.
- Know modern Lakehouse architectures, Delta Lake, Structured Streaming, and cloud infrastructure.
- Be able to independently architect and deliver complex, large-scale data solutions with minimal supervision.
- Demonstrate analytical and problem-solving skills, initiative, and the ability to make technical decisions autonomously.
- Be able to explain technical concepts and architectural decisions to business leaders and non-technical audiences.
- Be able to lead customer-facing technical engagements and build trusted customer relationships.
- Databricks Certified Data Engineer Professional certification is preferred.
- Experience with Delta Live Tables (DLT) and Unity Catalog (UC) is preferred.
- Consulting experience with large-scale data migrations, enterprise analytics transformations, or AI-assisted analytics initiatives is an advantage.
Responsibilities
- Lead customer engagements from discovery and technical assessment through architecture, implementation, and delivery.
- Design, implement, and optimize large-scale Apache Spark and Databricks workloads.
- Troubleshoot and optimize distributed data processing jobs and SQL workloads using knowledge of Spark internals.
- Develop and deliver proofs of concept, technical demonstrations, and customer workshops.
- Advise customers by identifying challenges and recommending data-driven solutions.
- Translate business requirements into practical, scalable data architectures and drive technical decisions.
- Collaborate with product and engineering teams to improve platform capabilities and technical solutions.
- Establish engineering best practices for PySpark, Databricks, and Lakehouse architecture.
- Mentor junior engineers and support data engineering teams’ technical development.
- Identify opportunities to improve performance, platform governance, and solution efficiency.
View Full Description & ApplyYou'll be redirected to the employer's site
