AI Evaluation Analyst

About the Role

micro1 is engaging AI Evaluation Analysts to contribute to a customer’s project focused on advancing frontier language model capabilities. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters. You will play a crucial role in producing evaluation and training data that directly influence how powerful AI models understand and interact. This i

What You'll Do

• Author detailed, task-based multi-turn conversations and rubrics aligned with project specifications.
• Test and refine conversation drafts against frontier large language models, iterating to meet quality and difficulty requirements.
• Deliver comprehensive evaluation assets including transcripts, target behaviors, binary rubrics, and supporting evidence.
• Ensure strict fidelity to evolving project specs while maintaining high throughput and attention to detail.
• Validate and calibrate outputs with team leads and quality control as guidelines change.
• Work independently and consistently, meeting expected output rates for deliverable completion.

You're a Good Fit If You

• Required skills: Working knowledge of frontier LLM behavior, Data Annotation, Written English clarity and structure, Spec fidelity at volume, Self-direction to a detailed spec
• Native-level written English with exceptional clarity, structure, and attention to detail.
• Prior experience in data annotation, RLHF, SFT, evaluation, or prompt engineering for AI systems.
• Working knowledge of frontier LLM behaviors and common model failure patterns.
• Demonstrated ability to interpret and apply highly detailed specifications without supervision.
• Strong critical thinking and analytical skills in writing-heavy or analysis-heavy domains.
• Experience authoring evaluation items, rubrics, or conducting deep analysis of technology outputs.
• Background in research, editorial, technical writing, or quality assurance is a plus.

Role Highlights

Pay & Payout

Engaged directly through micro1. Apply via the link below - micro1 handles onboarding and payment.

Opens micro1 in a new tab. You pay nothing; we may earn a referral fee.