Senior Software Engineer, Backend (Workload Safety)
New
A
AffirmCloud infrastructure
This remote role is open only to candidates residing in Alberta, British Columbia, Manitoba, New Brunswick, Newfoundland and Labrador, Nova Scotia, Ontario, Prince Edward Island, or Saskatchewan.Full-TimeSenior
SalaryCAN base pay range per year: CAD $153,000 - $213,000.
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of experience running Kubernetes in production at scale
- Required Skills
- PythonKubernetesGo
Requirements
- Have 5+ years of experience running Kubernetes in production at scale, ideally Amazon EKS across many clusters.
- Understand Kubernetes resource management and scheduling, including requests and limits, OOM and CPU throttling, affinity and anti-affinity, taints and tolerations, and topology spread constraints.
- Understand pod lifecycle and disruption management, including graceful termination, pod disruption budgets, and node draining.
- Have experience enforcing configuration standards through admission control and policy-as-code, such as OPA Gatekeeper or Kyverno.
- Be proficient in Go or Python, including building production infrastructure software such as Kubernetes controllers, operators, or admission webhooks.
- Demonstrate strong production debugging and the ability to turn recurring incidents into systemic fixes.
- Have led technical initiatives across multiple teams, driven adoption of shared standards, and mentored engineers.
- Have experience quantifying infrastructure changes’ impact on availability, latency, and cost.
- Have strong written communication skills, including writing standards and guidance teams can adopt independently.
Responsibilities
- Define and evolve workload safety standards for Affirm’s EKS fleet, including resource settings, workload isolation and placement, disruption tolerance, and graceful termination.
- Take standards from research and prototyping through early-adopter onboarding, self-service guidance, automation, and policy guardrails.
- Lead cross-team initiatives from design through fleet-wide rollout, mentor engineers, and uphold design, code review, and operational readiness standards.
- Isolate workloads by criticality and resource profile to protect critical paths such as checkout.
- Resolve stuck workloads and node lifecycle issues, including termination, pod disruption budget, and Karpenter problems.
- Determine and maintain safe cluster operating limits, and develop control plane scaling patterns for bursty workloads.
- Analyze fleet-wide CPU, memory, disk, and network profiles to guide EC2 instance type strategy and automate the analysis.
- Measure standards’ effects on availability, latency, risk, and cost to prioritize the roadmap.
- Participate in on-call for Kubernetes runtime issues and turn recurring failures into roadmap items.
- Work with teams running services, batch jobs, and ML workloads on EKS, as well as deployment, capacity, and SRE teams.
View Full Description & ApplyYou'll be redirected to the employer's site
