Senior Collections Data Engineer
New
I
iCOUNTERCybersecurity threat intelligence
CA - RemoteFull-TimeSenior
Salary$140K - $180K; $140K – $180K • Offers Equity; Estimated Base Salary $140K – $180K • Offers Equity
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of experience as a Software Engineer or Data Engineer
- Required Skills
- PythonCybersecurity
Requirements
- Bring 5+ years of experience as a Software Engineer or Data Engineer, focused on large-scale data collection and processing.
- Have deep experience collecting data at scale from open and dark web sources, including scraping, crawling, API integration, Tor-based collection, and anti-detection or anti-bot techniques.
- Have strong expertise building and operating data pipelines; Airflow, Dagster, Prefect, Spark, Kafka, Flink, or equivalent are listed.
- Demonstrate advanced data parsing skills, including converting messy, unstructured web data into structured intelligence.
- Bring expert-level Python proficiency, including async, concurrency, and performance optimization.
- Have hands-on experience implementing agentic workflows and integrating LLMs into production data systems; LangChain, CrewAI, AutoGen, LlamaIndex, or similar frameworks are listed.
- Have a track record of building and maintaining production APIs; FastAPI, Flask, or similar are listed.
- Have a strong understanding of cybersecurity concepts and threat intelligence data types, including IOCs, TTPs, malware samples, phishing kits, and underground forums.
- Experience with cloud platforms is sought.
- Knowledge of modern data stack tools such as dbt, Snowflake, ClickHouse, or DuckDB is sought.
- Familiarity with Docker, Kubernetes, and infrastructure-as-code is sought.
- Prior work in cybersecurity, threat intelligence, or intelligence collection organizations is sought.
Responsibilities
- Design and scale high-volume data collection systems for open-web, deep-web, and dark-web sources, including Tor, I2P, and other anonymity networks.
- Build and maintain fault-tolerant batch and streaming pipelines for data ingestion, transformation, enrichment, and storage.
- Develop parsing and normalization logic for unstructured, semi-structured, and evolving threat data formats.
- Design, build, and maintain autonomous, LLM-powered agentic workflows for multi-step collection and enrichment tasks.
- Architect and operate media pipelines to ingest, process, store, and enrich images, videos, screenshots, and documents.
- Design and build internal and external APIs for downstream security teams and platforms.
- Write production-grade Python for data engineering and automation.
- Collaborate with threat intelligence analysts, detection engineers, and security researchers to translate intelligence requirements into data systems.
- Monitor, troubleshoot, and optimize collection performance, data quality, and latency.
View Full Description & ApplyYou'll be redirected to the employer's site
