AI Quality Analyst.

Our client, a global consulting firm, is hiring an AI Quality Analyst to join the team on a contract basis in Dublin, Ireland. The successful candidate will apply their legal, regulatory and compliance expertise to complex policy-enforcement decisions, evaluating both human reviewer performance and AI-generated outputs.

Responsibilities

• Evaluate and classify content decisions and AI-generated outputs across text, image and video, ensuring accuracy, consistency and quality.
• Apply strong critical thinking and subject-matter expertise to complex, sensitive and ambiguous cases.
• Participate in calibration and team collaboration sessions to ensure consistent interpretation and application of policies.
• Assess the quality of human reviewer decisions and provide clear, structured and actionable feedback to improve performance.
• Evaluate AI-generated outputs against defined quality, safety and compliance standards, providing structured feedback to support model improvement.
• Develop and maintain high-quality gold-standard datasets for training, benchmarking and measuring human and AI performance.
• Identify errors, recurring trends, inconsistencies and areas of ambiguity, recommending improvements to policies, guidelines and evaluation frameworks.
• Clearly communicate findings and decision-making, confidently highlighting errors, risks and areas requiring further review.
• Contribute to the ongoing development of Trust & Safety policies, quality standards and evaluation processes as technology, regulations and organisational requirements evolve.

Skillset

• At least 4 years of professional experience in compliance, legal, regulatory affairs, Trust & Safety, risk management, policy enforcement or a related discipline.
• Strong knowledge of AI governance, Responsible AI, Online Safety and content policy enforcement.
• Practical understanding of key regulatory frameworks, including the EU AI Act, GDPR and Online Safety legislation.
• Proven ability to interpret and apply complex policies, regulations, standards and legal requirements to real-world scenarios.
• Excellent critical thinking and analytical skills, with the ability to assess complex, nuanced and ambiguous cases.
• Experience conducting quality assessments, identifying errors and providing clear, evidence-based feedback.
• Strong written and verbal communication skills, with the ability to clearly articulate decisions, findings and recommendations.
• Exceptional attention to detail, with a consistent, objective and evidence-based approach to decision-making.
• Experience working with AI-generated content, AI systems, content policies or technology risk is highly desirable.
• Comfortable working collaboratively in calibration sessions and contributing to the development and refinement of guidelines, policies and quality frameworks.