Senior Software Engineer
New
L
LanceDBData Infrastructure
Americas or APAC timezones, Americas or APAC timezonesFull-TimeSenior
Salary$180K - $250K; $180K – $250K • Offers Equity
Apply NowOpens the employer's application page
Job Details
- Experience
- 10+ years
- Required Skills
- JavaC++RustSparkScalaDistributed Systems
Requirements
- 10+ years of experience building high-performance databases, big data systems, or large-scale data services
- Deep understanding of internals of open-source Big Data or AI training systems (e.g., Hadoop, Spark, Flink, Ray, Iceberg, Delta Lake, Hudi, ClickHouse, Trino, Presto, PyTorch, or JAX)
- Strong experience with high-performance computing in C++, Java, and/or Scala
- Experience with Rust (or willingness to learn it)
- Proven ability to move fast, work independently, and collaborate with a high-caliber team
Responsibilities
- Design and maintain efficient distributed Lance dataset operations
- Build efficient indices to enable predicate pushdown and accelerate queries in Spark, Ray, or Trino
- Develop table formats, data encodings, and various aspects of the Lance format in Rust
- Drive open-source community efforts to integrate the Lance format with Spark, Hive Metastore, Presto, Trino, Ray, and other data infrastructure systems
- Operate and improve internal data processing infrastructure
- Promote the Lance format in open-source communities and at Big Data conferences
View Full Description & ApplyYou'll be redirected to the employer's site
