Software Engineer, Backend (Platform)
New
V
VannevarNational security software
United StatesFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 4+ years of backend engineering experience
- Required Skills
- PythonETL
Requirements
- Have 4+ years of backend engineering experience.
- Have production experience with web scraping, data ingestion, ETL, or third-party integrations.
- Write production software in Python.
- Be familiar with relational databases and API design.
- Know how to maintain data quality across large, changing datasets.
- Have designed systems that tolerate upstream failures and changing external interfaces.
- Make tradeoffs among reliability, scalability, performance, maintainability, and cost.
- Communicate technical decisions clearly and work across engineering, product, and mission teams.
- Experience with distributed task processing, workflow orchestration, or large-scale data pipelines is a plus.
- Experience monitoring data quality and investigating production regressions is a plus.
Responsibilities
- Build and operate production web scrapers, third-party integrations, and data pipelines.
- Design systems for schema changes, rate limits, authentication requirements, and partial failures.
- Build retry, backfill, deduplication, idempotency, and replay mechanisms.
- Define and monitor data completeness, correctness, freshness, duplication, and provenance.
- Develop APIs, databases, and shared services for collection workflows.
- Partner with mission teams, platform teams, and product managers to evaluate sources and support new data requirements.
- Influence architecture, technology selection, and engineering standards for collection systems.
View Full Description & ApplyYou'll be redirected to the employer's site
