Staff Software Engineer, Search & Retrieval Infrastructure
New
S
SylloLegal AI
United States - RemoteFull-TimeStaff
Salary$190,000 - $230,000 USD
Apply NowOpens the employer's application page
Job Details
- Experience
- 8+ years
- Required Skills
- PythonElasticSearchJavaKafkaKubernetesGoRust
Requirements
- 8+ years of software engineering experience, with a proven track record at the Staff/Principal level scaling distributed, high-throughput systems.
- Deep, production-level expertise tuning and scaling Lucene-based search engines (Elasticsearch, Solr) and vector indexing infrastructure.
- Understanding of index internals, chunking strategies, and embedding retrieval optimization.
- Strong history of managing compute vs. storage trade-offs and designing cost-effective cold-storage scanning solutions.
- Extensive experience with complex data pipelines, high-throughput event streaming (Kafka, Kinesis), and distributed compute architectures.
- Expert command of cloud primitives (GCP preferred), Kubernetes, and infrastructure-as-code.
- Expert-level proficiency in systems-level and backend languages (Go, Rust, Python, or Java/C++).
Responsibilities
- Lead the optimization and architectural evolution of existing hybrid search infrastructure using Elasticsearch, Lucene, and dense vector databases.
- Design and implement intelligent, cost-effective data tiering strategies across hot, warm, and cold storage.
- Evolve distributed pipelines to execute asynchronous, massive-scale scans of petabytes of data.
- Drive down latency and cost-to-serve by analyzing bottlenecks and tuning indexing and querying algorithms.
- Act as the domain expert for the indexing and search ecosystem and guide teams on high-volume data modeling.
- Ensure fault-tolerant, highly available operations during massive ingest events and concurrent querying.
View Full Description & ApplyYou'll be redirected to the employer's site
