AI Evaluation Specialist (Remote Contractor)

About the role

We are looking for an AI Evaluation Specialist to work remotely on a contract basis. In this role you will help train next‑generation AI systems by providing high‑quality, real‑world feedback on model outputs. No prior AI experience is required; your domain knowledge and critical eye are what matter.

Key responsibilities

• Review AI‑generated text, images, or other content for quality, clarity, usability and overall user experience.
• Identify inconsistencies, weaknesses and improvement opportunities across diverse content types.
• Apply structured evaluation guidelines and deliver nuanced feedback to inform model improvements.
• Collaborate with cross‑functional teams to define quality standards, scoring criteria and review processes.
• Maintain high accuracy and consistency across evaluations to ensure reliable, actionable results.
• Contribute to the continuous enhancement of evaluation workflows and best practices.

Required profile

• Exceptional reading comprehension and acute attention to detail.
• Strong observational skills with excellent aesthetic or editorial judgment.
• Ability to think critically and deliver consistent, high‑quality evaluations in ambiguous scenarios.
• Excellent written and verbal communication skills.
• Comfort working across various content types and evaluation guidelines.
• Strong sense of ownership, reliability and commitment to delivering high‑quality work.
• Open‑minded approach to learning and quickly adapting to new workflows.

Required skills

• Familiarity with annotation, QA, moderation or evaluation workflows.
• Experience reviewing creative, visual or AI‑generated content.
• Knowledge of AI tools or large language models (LLMs) is a plus.