Member of Technical Staff, Training Performance Engineer

View full description

Requirements:

• Extremely strong software engineering skills.
• Proficiency in Python and related ML frameworks such as JAX, Pytorch and XLA/MLIR.
• Experience writing kernels for GPUs using CUDA, triton, etc
• Experience using large-scale distributed training strategies.
• Familiarity with autoregressive sequence models, such as Transformers.

Responsibilities:

• Design and write high-performant and scalable software for training.
• Understand architectural modifications and design choices and their effects on training throughput and quality.
• Write low-level CUDA, triton kernels to squeeze every last bit of performance from our accelerators.
• Research, implement, and experiment with ideas on our supercompute and data infrastructure.
• Learn from and work with the best researchers in the field.