Machine Learning Engineer - Distributed ML Systems
New
P
Pluralis ResearchMachine Learning
Competitive base salary for senior engineering roles in AustraliaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- PythonMachine LearninggRPCDistributed Systems
Requirements
- Strong experience building and operating distributed systems in production.
- Hands-on expertise with distributed training frameworks (FSDP, DeepSpeed, Megatron, or similar).
- Deep understanding of model parallelism (data, tensor, pipeline parallelism).
- Expert-level Python with production experience (concurrency, error handling, retry logic, clean architecture).
- Strong networking fundamentals: P2P systems, gRPC, routing, NAT traversal, distributed coordination.
- Experience optimizing GPU workloads, memory management, and large-scale compute efficiency.
Responsibilities
- Design and implement large-scale distributed training systems optimized for heterogeneous hardware operating under low-bandwidth, high-latency conditions.
- Develop and optimize model-parallel training strategies (data, tensor, pipeline parallelism) with custom sharding techniques that minimize communication overhead.
- Optimize GPU utilization, memory efficiency, and compute performance across distributed nodes.
- Implement robust checkpointing, state synchronization, and recovery mechanisms for long-running, fault-prone training jobs.
- Build monitoring and metrics systems to track training progress, model quality, and system bottlenecks.
- Architect resilient training systems where nodes can fail, networks can partition, and participants can dynamically join or leave.
- Design and optimize peer-to-peer topologies for decentralized coordination across non-co-located nodes.
- Implement NAT traversal, peer discovery, dynamic routing, and connection lifecycle management.
- Profile and optimize communication patterns to reduce latency and bandwidth overhead in multi-participant environments.
View Full Description & ApplyYou'll be redirected to the employer's site
