Senior Data Engineer
New
J
JobgetherData Engineering
100% remote work within the United StatesFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 6+ years
- Required Skills
- PythonGCPKafkaSnowflakeSparkBigQueryDatabricks
Requirements
- 6+ years of proven enterprise-level data engineering experience.
- Strong hands-on expertise in Python and Apache Spark.
- Advanced experience with Google Cloud Platform (GCP), particularly BigQuery.
- Practical knowledge of Apache Iceberg, Delta Lake, and Iceberg UniForm.
- Experience implementing Delta Sharing and building Kafka-based CDC and event-driven data pipelines.
- Hands-on experience with Snowflake Horizon Catalog and Databricks Unity Catalog.
- Solid understanding of modern data lake and lakehouse architectures.
- Experience designing scalable, reusable, and maintainable data integration solutions.
- Strong understanding of data lineage, metadata, governance, and access controls.
- Excellent written and verbal communication skills for cross-functional collaboration.
- Strong problem-solving skills with the ability to work independently.
Responsibilities
- Design and implement the Delta Lake write layer using Iceberg UniForm to enable multi-platform data consumption.
- Build scalable GCS-to-BigQuery ingestion pipelines for structured operational datasets.
- Develop robust data engineering solutions using Python and Apache Spark.
- Implement Kafka-based change data capture (CDC) patterns for real-time and near-real-time data ingestion.
- Design reusable, modular data-sharing adapters and components for lakehouse integrations.
- Develop data lineage, dependency tracking, and metadata capabilities for data governance.
- Configure Snowflake Horizon external tables to enable secure, zero-copy data access.
- Implement and certify Delta Sharing endpoints for Databricks interoperability.
- Build governed data access components, including RBAC and tenant-scoped authorization.
View Full Description & ApplyYou'll be redirected to the employer's site
