Senior Data Engineer, Emerging Products

New
N
NinjaOneData platform
We are flexible on remote working from home, if you are located in the USA and reside in one of the following states - CA, CO, CT, FL, GA, IL, KS, MA, MD, ME, NJ, NC, NY, OH, OR, TN, TX, VA, and WA.Full-TimeSenior
Salary140,000 - 200,000 USD per year
Apply NowOpens the employer's application page

Job Details

Experience
10+ years of experience in data engineering with a strong focus on data platform and distributed systems.
Required Skills
PythonApache AirflowApache KafkaSparkDatabricks

Requirements

  • Have 10+ years of data engineering experience focused on data platforms and distributed systems.
  • Have hands-on experience building and managing lakehouse architectures, including Medallion layering patterns.
  • Have deep expertise in Apache Kafka, Apache Spark, and Apache Airflow for streaming and batch pipeline development.
  • Be proficient in open table formats such as Iceberg, Delta Lake, or Hudi, and evaluate trade-offs among formats.
  • Have experience with Databricks for large-scale data processing and analytics workloads.
  • Be proficient in Python and SQL.
  • Have a bachelor’s degree in Computer Science, Computer Engineering, Information Technology, or equivalent work experience; preferred.
  • Hands-on experience with Trino or Starburst is a standout qualification.
  • Familiarity with hybrid streaming/batch architectures and event-driven data systems is a standout qualification.
  • Experience designing Gold-layer datasets for business intelligence and self-serve analytics is a standout qualification.

Responsibilities

  • Design and implement Medallion-layer data architectures with Bronze, Silver, and Gold layers.
  • Build structured ingestion pipelines and curated data layers for analytics and data science workloads.
  • Build and maintain scalable streaming and batch pipelines using Kafka, Spark, and Airflow.
  • Manage Iceberg, Delta, and Hudi table formats, selecting formats for workloads and maintaining performance and reliability.
  • Optimize large-scale analytics queries in Trino/Starburst and Databricks environments.
  • Monitor pipeline health and troubleshoot data quality and performance issues.
  • Improve the reliability and efficiency of the data platform.
  • Partner with data scientists, analysts, and product teams to understand data requirements and deliver platform solutions.
View Full Description & ApplyYou'll be redirected to the employer's site
140,000 - 200,000 USD per year
Apply Now