Senior Data Engineer
New
Z
ZipBuy Now, Pay Later
Remote-first opportunity for US-based employees with the option to work in-person out of our Manhattan office.Full-TimeSenior
SalaryThe Pay Range for this position: $134,000 - $175,000 USD
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of experience designing, building, and operating scalable data pipelines and enterprise data platforms in production environments.
- Required Skills
- PythonSQLApache AirflowSnowflakeApache KafkaAzureCI/CDData modelingdbtDatabricks
Requirements
- Have 5+ years of experience designing, building, and operating scalable data pipelines and enterprise data platforms in production.
- Bring advanced SQL and strong Python skills, including experience developing maintainable, tested, production-quality code.
- Have hands-on experience with a modern cloud data platform such as Snowflake, Databricks, Microsoft Fabric, BigQuery, or Redshift.
- Understand dimensional modeling, including fact and dimension tables, slowly changing dimensions, and reusable analytical data models.
- Have experience building ETL/ELT workflows with dbt or equivalent and orchestration tools such as Airflow or Dagster.
- Have experience with streaming, change data capture (CDC), or event-driven architectures using Kafka, Azure Event Hubs, Spark Structured Streaming, or comparable technologies.
- Understand data warehouses, data lakes, lakehouses, and curated data products.
- Have experience with data quality, automated testing, monitoring, observability, metadata, lineage, and governance practices.
- Have hands-on experience with a major cloud provider; Azure is preferred, and AWS and GCP experience are transferable.
- Have software engineering experience with Git, code reviews, CI/CD, infrastructure automation, and production troubleshooting.
- Be able to make architecture decisions and evaluate tradeoffs affecting scalability, performance, reliability, and cost.
- Have experience translating business requirements into scalable technical solutions in collaboration with Engineering, Analytics, Product, Risk, Finance, and Data Science.
- Have demonstrated ability to mentor engineers and establish reusable engineering standards.
- Be familiar with AI-enabled engineering tools and interested in data platforms supporting machine learning and intelligent applications.
Responsibilities
- Design, build, and operate scalable batch, streaming, and event-driven data pipelines.
- Integrate transactional systems, APIs, third-party providers, and enterprise data sources into the cloud data platform.
- Develop reusable data products and dimensional and semantic models for analytics, risk, finance, product, and operational decision-making.
- Build and optimize ETL/ELT workflows using transformation and orchestration frameworks.
- Improve data quality, reliability, and observability through validation, reconciliation, freshness monitoring, anomaly detection, and alerting.
- Design near-real-time data capabilities for transaction monitoring, fraud and risk signals, and operational intelligence.
- Improve platform performance and cost efficiency and establish scalable architecture patterns across warehouse and lakehouse environments.
- Build shared frameworks, self-service capabilities, and automated workflows to reduce repetitive engineering effort.
- Implement CI/CD, automated testing, infrastructure as code, documentation, governance, and production ownership practices.
- Partner with Data Science and Machine Learning teams on reliable, AI-ready datasets and pipelines; lead initiatives and mentor engineers.
View Full Description & ApplyYou'll be redirected to the employer's site
