ML Engineer — Evaluation

About the Role

Scale AI is seeking an ML Engineer to build automated evaluation systems for large language models. You will develop benchmarks, create model comparison tools, and design metrics that accurately measure model capabilities across diverse tasks.

Requirements

• 3+ years of ML engineering experience
• Strong skills in Python and statistical analysis
• Experience with LLM evaluation methodologies
• Understanding of NLP benchmarks and metrics
• Strong software engineering fundamentals

Nice to Have

• Experience building leaderboards or benchmark platforms
• Background in psychometrics or test design
• Familiarity with Scale AI's evaluation platform
• Experience with multi-turn evaluation frameworks

Benefits

Skills

Related Jobs