Software Engineer, Data Infrastructure
New
C
CohereAI Infrastructure
New York, Toronto, San Francisco, Montreal, Canada, United StatesFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 4+ years
- Required Skills
- PythonKubernetesSparkDistributed Systems
Requirements
- 4+ years of experience working on data storage infrastructure
- Strong command of Python
- Kubernetes experience, especially on the storage side (Persistent Volumes, CSI drivers, etc.)
- Ability to transform unstructured data into performant datasets
- Experience with distributed data processing frameworks such as Apache Beam, Spark, or Flink
- Genuine excitement about AI
- Comfort operating at the edge of known technology
Responsibilities
- Build and maintain high-performance data layers for modeling teams.
- Work on petabyte-scale storage infrastructure.
- Address networking and performance challenges related to data storage.
- Collaborate with researchers and engineers to support training and evaluation jobs.
- Transform unstructured data into performant datasets.
- Manage storage backends including S3, GCS, and POSIX.
View Full Description & ApplyYou'll be redirected to the employer's site
