Senior Software Engineer (Data Platform)

Posted 2024-07-11

💎 Seniority level: Senior, Progressive experience in all listed areas

📍 Location: San Francisco Bay Area, Seattle, India, UK

💸 Salary: 150000 - 180000 USD per year

🔍 Industry: B2B technology

🗣️ Languages: English

⏳ Experience: Progressive experience in all listed areas

🪄 Skills: Data AnalysisGitKafkaYarnAirflowClickhouseData analysisSpark

Four-year degree in Computer Science, or related field OR equivalent experience.
Progressive experience in designing frameworks and writing efficient data pipelines, including batches and real-time streams.
Understanding of data strategies, articulate data analysis & data model design, and evolve data products according to business requirements.
Experience with the Spark Ecosystem (YARN, Executors, Livy, etc).
Experience in large scale data streaming, particularly Kafka or similar technologies (Pulsar, Kinesis, etc).
Experience with data orchestration frameworks, particularly Airflow or similar.
Experience with columnar data stores, particularly Parquet and Clickhouse.
Strong SDLC principles (CI/CD, Unit Testing, git, etc).
General understanding of AWS EMR, EC2, S3.

Build out all aspects of the Demandbase Data ecosystem and move products from R&D into production scale.
Design and build data pipelines to create the next generation of Demandbase’s Unified Data Platform.
Work across the data stack to build and productionalize data pipelines for massive amounts of data.
Build DAGs in Airflow for orchestration and monitoring of data pipelines.

Posted 2024-09-07

📍 United States, India, United Kingdom

🧭 Full-Time

💸 150000 - 180000 USD per year

🔍 B2B Technology

Four-year degree in Computer Science, or related field OR equivalent experience.
Progressive experience in understanding data strategies and evolving data products.
Experience designing and building low latency analytics APIs at scale.
Proficiency in at least one JVM language (Java, Scala, Kotlin, etc.).
Familiarity with the Spark Ecosystem (YARN, Executors, Livy, etc.).
Experience with data orchestration frameworks, particularly Airflow or similar.
Knowledge of columnar data stores, particularly Parquet and StarRocks.
Strong SDLC principles (CI/CD, Unit Testing, git, etc.).
General understanding of AWS EMR, EC2, S3.

Design and build the next generation of Demandbase’s Unified Data Platform through a combination of data pipelines, APIs, internal tools and 3rd party/open source tooling.
Work across the data stack to build and productionalize data pipelines for massive amounts of data.
Integrate 3rd party and open source tools into our data platform to build out various aspects of our data platform.
Build DAGs in Airflow for orchestration and monitoring of data pipelines.

AWSData AnalysisGitJavaJVMKafkaKotlinYarnAirflowData analysisSparkCI/CD

Posted 2024-09-07

🔧 Requirements