Member of Engineering - Evaluations
P
PoolsideArtificial General Intelligence
Remote (EMEA/East Coast)Full-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- AWSPythonGCPKafkaAzureGrafanaPrometheusDatadog
Requirements
- Strong engineering background
- Experience leading software projects cross functionally
- Experience building highly reliable and well tested services
- Experience with distributed systems
- Experience with data pipelines
- Experience with distributed processing
- Experience with message queues
- Experience with event-driven architectures, e.g. Kafka, Google pub/sub
- Experience with clouds: managed services, storage (GCP, AWS, Azure)
- Experience with monitoring and alerting: Grafana, Prometheus, Datadog
- A product mindset towards building developer facing software
- A knack for collaborating with researchers and ML engineers to identify opportunities
Responsibilities
- Build a scalable self-serve evaluation platform to power our research and development
- Design a Python framework that makes it easy for poolsiders to implement both internal and public benchmarks in a centralized way
- Build and maintain the pipeline that runs distributed evaluations at scale
- Collaborate with modeling and product teams to identify opportunities to improve our experimentation and evaluation tooling
View Full Description & ApplyYou'll be redirected to the employer's site
