Senior Data Engineer | Emerging Products
New
J
JobgetherIT Security
United StatesFull-TimeSenior
Salary110,000 - 200,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 10+ years
- Required Skills
- PythonSQLApache AirflowApache KafkaSparkData modelingDatabricks
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering, IT, or equivalent experience.
- 10+ years of experience in data engineering at an enterprise scale.
- Proven experience designing lakehouse architectures and Medallion data models.
- Strong hands-on expertise with Apache Kafka, Apache Spark, and Apache Airflow.
- Advanced knowledge of Apache Iceberg, Delta Lake, or Hudi.
- Experience working with Databricks for large-scale processing.
- Strong programming proficiency in Python and SQL.
- Knowledge of event-driven architectures and hybrid streaming/batch systems.
- Solid understanding of data modeling and performance optimization.
- Excellent communication and collaboration skills.
Responsibilities
- Design and implement scalable lakehouse architectures using Bronze, Silver, and Gold patterns.
- Build and maintain high-performance streaming and batch data pipelines using Kafka, Spark, and Airflow.
- Develop and manage open table format solutions, including Iceberg, Delta Lake, and Hudi.
- Optimize distributed query environments and analytics platforms to improve performance and scalability.
- Manage platform reliability by monitoring pipeline health and troubleshooting data quality.
- Architect and maintain solutions supporting hybrid streaming and batch processing environments.
- Partner with data scientists, analysts, and product teams to deliver platform capabilities.
- Create and maintain Gold-layer datasets for business intelligence and self-service analytics.
View Full Description & ApplyYou'll be redirected to the employer's site
