Remoote.app

Work from anywhere. Start here.

JobsFree resume builderCompaniesLog in
Apply

Senior Software Engineer (Data Platform)

Posted 2024-09-07

View full description

💎 Seniority level: Senior, Progressive experience in all required areas

📍 Location: USA, India, UK

💸 Salary: 150000 - 180000 USD per year

🔍 Industry: B2B Technology

🗣️ Languages: English

⏳ Experience: Progressive experience in all required areas

🪄 Skills: AWSData AnalysisGitJavaJVMKafkaKotlinYarnAirflowData analysisSparkCI/CD

Requirements:
  • Four-year degree in Computer Science, or related field OR equivalent experience.
  • Understanding of data strategies and ability to articulate data analysis & data model design, evolving data products according to business requirements.
  • Experience in designing and building low latency analytics APIs at scale.
  • Proficiency in at least one JVM language (Java, Scala, Kotlin, etc.).
  • Knowledge of the Spark Ecosystem (YARN, Executors, Livy, etc.).
  • Experience with data orchestration frameworks, particularly Airflow.
  • Familiarity with columnar data stores, particularly Parquet and StarRocks.
  • Strong understanding of SDLC principles (CI/CD, Unit Testing, git, etc.).
  • General understanding of AWS EMR, EC2, S3.
Responsibilities:
  • Design and build the next generation of Demandbase’s Unified Data Platform through a combination of data pipelines, APIs, internal tools and 3rd party/open source tooling.
  • Work across the data stack to build and productionalize data pipelines for massive amounts of data.
  • Integrate 3rd party and open source tools into our data platform to build out various aspects of our data platform.
  • Build DAGs in Airflow for orchestration and monitoring of data pipelines.
Apply

Related Jobs

Apply
🔥 Senior Software Engineer - Data Platform
Posted 2024-11-07

📍 United States

🔍 Life sciences

  • Applicants must have the unrestricted right to work in the United States.
  • Veeva will not provide sponsorship at this time.

  • Spearhead the development of new architecture for the Data platform from the ground up.
  • Design and build a resilient, scalable cloud-based platform along with its accompanying tools.
  • Empower Opendata teams to efficiently create and distribute valuable data assets.
  • Exercise end-to-end ownership for the project.

Backend DevelopmentLeadershipSoftware DevelopmentCross-functional Team LeadershipCommunication SkillsAnalytical SkillsCollaboration

Posted 2024-11-07
Apply
Apply
🔥 Senior Software Engineer - Data Platform
Posted 2024-11-07

📍 United States

🔍 Life sciences

  • Applicants must have the unrestricted right to work in the United States.
  • Veeva will not provide sponsorship at this time.

  • Spearhead the development of entirely new architecture for Veeva's Data platform from the ground up.
  • Design and build a resilient, scalable cloud-based platform along with its accompanying tools.
  • Empower Opendata teams to efficiently create and distribute valuable data assets.
  • End-to-end ownership of projects, guiding the course of action and executing solutions creatively.

Backend DevelopmentSoftware DevelopmentCloud ComputingData AnalysisGitJavaJavascriptSoftware ArchitectureJavaScriptData analysis

Posted 2024-11-07
Apply
Apply
🔥 Senior Software Engineer (Data Platform)
Posted 2024-10-24

📍 USA

🧭 Full-Time

💸 169000 - 240000 USD per year

🔍 Financial services

  • 5+ years of industry experience in building large scale production systems.
  • Experience building and owning large-scale stream processing systems.
  • Experience building and operating robust and highly available infrastructure.
  • Working knowledge of Relational and NoSQL databases.
  • Experience working with Data Warehouse solutions.
  • Experience with industry standard stream processing frameworks like Spark, Samza, Flink, Beam etc.
  • Experience leading technical projects and mentoring junior engineers.
  • Exceptionally collaborative with a history of delivering complex technical projects and working closely with stakeholders.
  • This position requires either equivalent practical experience or a Bachelor’s degree in a related field.

  • Help support the Data Platform that forms the backbone for several thousands of offline workloads at Affirm.
  • Design and build data infrastructure systems, services, and tools to handle new Affirm products and business requirements that securely scale over millions of users and their transactions.
  • Build frameworks and services which will be used by other engineering teams at Affirm to manage billions of dollars in loans and power customer experiences.
  • Improve the reliability and efficiency of the Data Platform at scale and high reliability.
  • Engage other teams at Affirm about their use of the Data platform to ensure we are always building the right thing.

Backend DevelopmentLeadershipSoftware DevelopmentSQLData AnalysisElasticSearchKafkaCross-functional Team LeadershipApache KafkaData analysisElasticsearchSparkCollaboration

Posted 2024-10-24
Apply
Apply
🔥 Senior Software Engineer (Data Platform)
Posted 2024-07-11

📍 San Francisco Bay Area, Seattle, India, UK

🧭 Full-Time

💸 150000 - 180000 USD per year

🔍 B2B technology

  • Four-year degree in Computer Science, or related field OR equivalent experience.
  • Progressive experience in designing frameworks and writing efficient data pipelines, including batches and real-time streams.
  • Understanding of data strategies, articulate data analysis & data model design, and evolve data products according to business requirements.
  • Experience with the Spark Ecosystem (YARN, Executors, Livy, etc).
  • Experience in large scale data streaming, particularly Kafka or similar technologies (Pulsar, Kinesis, etc).
  • Experience with data orchestration frameworks, particularly Airflow or similar.
  • Experience with columnar data stores, particularly Parquet and Clickhouse.
  • Strong SDLC principles (CI/CD, Unit Testing, git, etc).
  • General understanding of AWS EMR, EC2, S3.

  • Build out all aspects of the Demandbase Data ecosystem and move products from R&D into production scale.
  • Design and build data pipelines to create the next generation of Demandbase’s Unified Data Platform.
  • Work across the data stack to build and productionalize data pipelines for massive amounts of data.
  • Build DAGs in Airflow for orchestration and monitoring of data pipelines.

Data AnalysisGitKafkaYarnAirflowClickhouseData analysisSpark

Posted 2024-07-11
Apply