Databricks Architect - US Based Remote

View full description

Requirements:

• 7+ years’ experience in data engineering or software development, focusing on data architecture.
• Extensive experience (3-5+ years) with Databricks and the Medallion Architecture.
• Expertise in Python, including developing production-grade data pipelines and reusable automation scripts.
• Proficiency in PySpark and SQL with experience integrating data sources with Databricks.
• Expertise in cloud platform services (AWS, Azure, GCP) and their data storage solutions.
• In-depth experience with Delta Lake for ACID transactions and data processing.
• Strong understanding of data pipelines and transformation workflows.
• Experience building automation tools or reusable scripts.
• Knowledge of Alteryx and Tableau integration is a plus.

Responsibilities:

• Design and implement scalable, high-performance data architectures using Databricks and the Medallion Architecture.
• Lead requirement-gathering sessions with clients and internal teams to understand business needs and define best practices for solutions including data workflows.
• Collaborate with cloud platform teams for data storage optimization.
• Translate complex data processes into optimized Databricks workflows using PySpark and SQL.
• Develop reusable automation scripts for workflow migrations.
• Provide hands-on development and troubleshooting support.
• Partner with teams to establish data governance frameworks and best practices.
• Deliver training and documentation to enhance data literacy.