Apply

Senior Software Engineer (Data Platform)

Posted 6 days agoViewed

View full description

💎 Seniority level: Senior

📍 Location: San Francisco Bay Area, Seattle, India, UK

💸 Salary: 150000.0 - 180000.0 USD per year

🔍 Industry: B2B technology

🗣️ Languages: English

🪄 Skills: AWSApache AirflowJavaKafkaKotlinSparkTerraformScalaData modeling

Requirements:
  • Four-year degree in Computer Science, or related field OR equivalent experience.
  • Understanding of data strategies, data analysis, and data model design.
  • Experience designing and building low latency analytics APIs.
  • Proficiency in at least one JVM language (Java, Scala, Kotlin, etc.).
  • Familiarity with the Spark Ecosystem (YARN, Executors, Livy, etc.).
  • Data orchestration frameworks, particularly Airflow or similar.
  • Experience with columnar data stores, particularly Parquet and StarRocks.
  • Strong SDLC principles (CI/CD, Unit Testing, git, etc.).
  • General understanding of AWS EMR, EC2, S3.
Responsibilities:
  • Design and build the next generation of Demandbase’s Unified Data Platform.
  • Develop data pipelines for ingestion, normalization, and analysis.
  • Integrate 3rd party and open source tools into the data platform.
  • Build DAGs in Airflow for orchestration and monitoring of data pipelines.
Apply

Related Jobs

Apply

📍 USA

🧭 Full-Time

💸 169000 - 240000 USD per year

🔍 Financial services

  • 5+ years of industry experience in building large scale production systems.
  • Experience building and owning large-scale stream processing systems.
  • Experience building and operating robust and highly available infrastructure.
  • Working knowledge of Relational and NoSQL databases.
  • Experience working with Data Warehouse solutions.
  • Experience with industry standard stream processing frameworks like Spark, Samza, Flink, Beam etc.
  • Experience leading technical projects and mentoring junior engineers.
  • Exceptionally collaborative with a history of delivering complex technical projects and working closely with stakeholders.
  • This position requires either equivalent practical experience or a Bachelor’s degree in a related field.

  • Help support the Data Platform that forms the backbone for several thousands of offline workloads at Affirm.
  • Design and build data infrastructure systems, services, and tools to handle new Affirm products and business requirements that securely scale over millions of users and their transactions.
  • Build frameworks and services which will be used by other engineering teams at Affirm to manage billions of dollars in loans and power customer experiences.
  • Improve the reliability and efficiency of the Data Platform at scale and high reliability.
  • Engage other teams at Affirm about their use of the Data platform to ensure we are always building the right thing.

Backend DevelopmentLeadershipSoftware DevelopmentSQLData AnalysisElasticSearchKafkaCross-functional Team LeadershipApache KafkaSparkCollaboration

Posted about 2 months ago
Apply