Senior Software Engineer – Open Source & SWE-Bench Evaluation
New
A
Anyone AISoftware engineering
Location: Argentina - Fully Remote
Secondary Locations: Uruguay, Chile, Ecuador - Fully Remote, Portugal, Mexico - Fully Remote, Colombia - Fully Remote, Spain, BrazilContractSenior
Salary65 USD per hour
Apply NowOpens the employer's application page
Job Details
- Experience
- 3+ years of professional software engineering experience
- Required Skills
- GitSoftware EngineeringDebuggingGitHubUnit Testing
Requirements
- Have 3+ years of professional software engineering experience.
- Have strong experience working with large, multi-file codebases.
- Have experience reviewing pull requests, debugging issues, and maintaining production code.
- Have a strong understanding of unit testing and test coverage.
- Be able to evaluate whether tests validate a solution without unnecessarily restricting implementation approaches.
- Have experience with dependency management, environment setup, and reproducibility.
- Have a strong understanding of Git and GitHub-based development workflows.
- Be able to analyze complex technical problems and provide clear written feedback.
- Open-source contributions or maintenance experience is a nice to have.
- Experience with SWE-Bench, SWE-Bench Verified, or similar coding benchmarks is a nice to have.
- Experience with Docker, CI/CD, pip, conda, or dependency pinning is a nice to have.
- Experience with AI/ML evaluation, data curation, RLHF, or benchmark development is a nice to have.
Responsibilities
- Review coding tasks derived from real GitHub issues and pull requests.
- Assess whether problem statements and success criteria are clear and complete.
- Evaluate unit tests for correctness, coverage, and robustness.
- Identify flaky tests, missing dependencies, version conflicts, and environment issues.
- Determine whether tasks can be reliably reproduced across environments.
- Assess the real-world difficulty and complexity of each task.
- Provide recommendations on whether tasks should be accepted, improved, or excluded.
View Full Description & ApplyYou'll be redirected to the employer's site
