Principal Software Engineer - Agentic SOC

New

HuntressCybersecurity

Remote US, OnlyFull-TimePrincipal

Salary$215,000 to $240,000 base plus bonus and equity

Apply NowOpens the employer's application page

More jobs in this role

Job Details

Experience: 15+ years of experience developing complex software products, including significant time as the technical lead on production systems
Required Skills: AWSRuby on RailsAzurePostgresDistributed Systems

Requirements

Bring 15+ years of experience developing complex software products, including significant time as the technical lead on production systems.
Have hands-on experience designing, shipping, and operating LLM-based agents in production, including tool use, context management, structured output, guardrails, and evaluation.
Understand common failure modes of LLM-based agents.
Have built evaluation for non-deterministic systems, such as labeled datasets, offline evaluations, and regression gates.
Have experience measuring system quality over time.
Have experience building systems that handle untrusted input, enforce multi-tenant isolation, and produce auditable trails.
Have experience with automated decisions that carry real consequences and judgment about what to automate versus leave to people.
Have deep backend expertise in distributed systems, queues, durable workflows, and concurrency.
Have a track record of designing for throughput and correctness under load.
Have strong skills in one or more backend languages and the ability to learn new ones; the primary stack is Ruby on Rails, and fluency is expected.
Have experience with AWS, Azure, or other public cloud environments.
Have experience with data stores such as Postgres and Redis.
Have experience with AI coding tools, such as Claude Code.
Have a BS or MS in Computer Science or Engineering, or equivalent experience.
Bonus: experience in or building for a SOC, incident response, threat hunting, or detection engineering; familiarity with endpoint and identity telemetry or attacker tradecraft; or experience adversarially testing ML or LLM systems.

Responsibilities

Own the architecture of agentic investigations, including signal flow, investigation state, agent tools, and investigation outputs.
Work with SOC analysts and product researchers to translate analyst triage and investigation practices into agent behavior.
Build LLM-powered preprocessing that summarizes, correlates, and highlights relevant information from analyst tools and data sources.
Design for untrusted data, tenant isolation, trust boundaries, and auditability.
Define evaluation methods for investigation correctness, reasoning, and documentation, and require evidence for prompt, model, and tool changes.
Use measured performance to decide which actions agents take autonomously and which require human review.
Build production systems for reliability, latency, and cost at scale, with traceable investigation conclusions and actions.
Design agent-to-analyst handoffs that help analysts work faster and stay informed.
Prototype uncertain technical approaches, derisk them, and hand off validated solutions.
Provide technical leadership through code review, pairing, shared patterns, and alignment with engineering, product, SOC leadership, and executives.

View Full Description & ApplyYou'll be redirected to the employer's site