Apply📍 San Francisco Bay Area, Seattle, India, UK
🧭 Full-Time
💸 150000 - 180000 USD per year
🔍 B2B technology
- Four-year degree in Computer Science, or related field OR equivalent experience.
- Progressive experience in designing frameworks and writing efficient data pipelines, including batches and real-time streams.
- Understanding of data strategies, articulate data analysis & data model design, and evolve data products according to business requirements.
- Experience with the Spark Ecosystem (YARN, Executors, Livy, etc).
- Experience in large scale data streaming, particularly Kafka or similar technologies (Pulsar, Kinesis, etc).
- Experience with data orchestration frameworks, particularly Airflow or similar.
- Experience with columnar data stores, particularly Parquet and Clickhouse.
- Strong SDLC principles (CI/CD, Unit Testing, git, etc).
- General understanding of AWS EMR, EC2, S3.
- Build out all aspects of the Demandbase Data ecosystem and move products from R&D into production scale.
- Design and build data pipelines to create the next generation of Demandbase’s Unified Data Platform.
- Work across the data stack to build and productionalize data pipelines for massive amounts of data.
- Build DAGs in Airflow for orchestration and monitoring of data pipelines.
Data AnalysisGitKafkaYarnAirflowClickhouseData analysisSpark
Posted 2024-07-11
Apply