Data Quality & AI Operations Manager

About the role

Extend is building a Cultural Intelligence Platform that combines media planning, influencer buying and brand monitoring with AI at its core. As the Data Quality & AI Operations Manager you will own the content and data operations, driving a leaner profiling process and increasing automation through AI fine‑tuning.

Key responsibilities

• Build and maintain golden sets and evaluation suites, defining acceptance criteria for each AI admin iteration.
• Curate training and evaluation data, run benchmarks, analyse failure modes and partner with engineering teams on model fine‑tuning and deployment.
• Design the human‑in‑the‑loop (HITL) mechanism, setting boundaries between human review and auto‑approval and feeding corrections back into prompts and training data.
• Author and maintain Arabic/English labeling guidelines and adjudicate edge cases requiring cultural and linguistic judgment.
• Lead the content operations team, overseeing rater training, workflow design, throughput and quality SLAs.
• Establish and track a quality metrics framework covering accuracy, automation rate, inter‑annotator agreement, cost and turnaround per profile.

Required profile

• Fluent in Arabic with strong knowledge of Arab media, brands, publishers and cultural context.
• Hands‑on experience with AI agents/LLM systems, including evaluation design, golden‑set creation and model feedback loops.
• At least 2 years in AI data operations or applied AI quality, preferably from a data‑labeling company or an in‑house AI annotation team.
• Comfortable collaborating across Riyadh and Shanghai time zones.

Required skills

• LLM evaluation design and golden‑set creation.
• Model fine‑tuning concepts (SFT, RLHF, prompt iteration).
• Experience with data‑labeling platforms such as Scale AI, Surge, Appen, Sama, Invisible, Labelbox.
• Human‑in‑the‑loop (HITL) workflow design.
• Arabic/English labeling guideline development.