Senior Software Engineer, Data Acquisition
P
People Data LabsData Acquisition
The permanent ability to work wherever and however you wantFull-TimeSenior
Salary$160K - $200K
Apply NowOpens the employer's application page
Job Details
- Experience
- 7+ years
- Required Skills
- PythonETLKafkaGoRustLinuxDistributed Systems
Requirements
- 7+ years of professional experience building or operating backend or infrastructure systems at scale
- Solid programming experience in Python, Go, Rust, or similar, including experience with async / await, coroutines, or concurrency frameworks
- Strong grasp of software architecture and backend fundamentals; you can reason clearly about concurrency, scalability, and fault tolerance
- Solid understanding of browser rendering pipeline, web application architecture (auth, cookies, http request / response)
- Familiarity with network architecture and debugging (HTTP, DNS, proxies, packet capture and analysis)
- Solid understanding of distributed systems concepts: parallelism, asynchronous programming, backpressure, and message-driven design
- Experience designing or maintaining resilient data ingestion, API integration, or ETL systems
- Proficiency with Linux / Unix command-line tools and system resource management
- Familiarity with message queues, orchestration, and distributed task systems (Kafka, SQS, Airflow, etc.)
- Experience evaluating and monitoring data quality, ensuring consistency, completeness, and reliability across releases
Responsibilities
- Contribute to the architecture and improvement of our data acquisition and processing platform, increasing reliability, throughput, and observability
- Use and develop web crawling technologies to capture and catalog data on the internet
- Build, operate, and evolve large-scale distributed systems that collect, process, and deliver data from across the web
- Design and develop backend services that manage distributed job orchestration, data pipelines, and large-scale asynchronous workloads
- Structure and model captured data, ensuring high quality and consistency across datasets
- Continuously improve the speed, scalability, and fault-tolerance of our ingestion systems
- Partner with data product and engineering teams to design and implement new data products
View Full Description & ApplyYou'll be redirected to the employer's site
