Project Overview

Join a growing community of professionals advancing the next wave of AI. As an AI Trainer, you’ll play a hands-on role by analyzing and providing feedback on data to improve LLM performance, helping ensure that the next generation of AI technology is accurate and trustworthy.

We are seeking a skilled Licensed US-Based Attorney to work as a project consultant in our AI Labor Marketplace. This is not a full-time employment position — you will be engaged as an expert project consultant on a contract basis.

Location: U.S.-based experts only

Engagement: Part-time, 1099 project-based expert evaluation work

Work Type: Remote

Project Summary

This project supports the development of an AI benchmarking framework designed to measure how effectively artificial intelligence models perform real-world, economically valuable knowledge work. The benchmark evaluates AI systems against tasks commonly performed by legal professionals in business environments.

Contributors will create expert benchmark outputs, define evaluation criteria, and assess AI-generated work products across a range of professional scenarios. The goal is to establish rigorous, expert-driven standards for measuring AI performance on complex business and regulatory workflows.

Consultant Engagement Terms

This is a project-based consultant role. Consultants will be paid on a per-project basis; hourly rates are estimates based on anticipated completion time. Consultants control their own schedule, provide their own tools, and may simultaneously provide services to other vendors/employers (subject to those vendors’ allowances).

Responsibilities

Contributors will:

• Create benchmark answers for realistic legal scenarios
• Develop scoring rubrics defining high-quality legal work product
• Evaluate AI-generated legal analyses and written deliverables
• Identify legal inaccuracies, unsupported conclusions, and material omissions
• Assess legal reasoning, risk identification, and practical applicability
• Provide written justifications for scores and evaluation decisions
• Participate in calibration and quality review exercises

Expected Outcomes

• High-quality benchmark answers reflecting professional legal practices
• Clear, defensible evaluation rubrics aligned with real-world business requirements
• Consistent and reliable assessment of AI-generated work products
• Identification of strengths, weaknesses, and risk areas in model performance
• Expert evaluations that support rigorous benchmarking of AI systems on economically valuable knowledge work
• Documentation that enables repeatable and scalable model evaluation

Qualifications

• Active U.S. law license in good standing
• Minimum 3+ years of professional legal experience
• Strong legal research, analysis, and written communication skills
• Ability to assess legal arguments for accuracy, completeness, and sound reasoning
• Experience reviewing contracts, policies, memoranda, legal correspondence, regulatory guidance, or legal analyses
• Strong attention to detail and professional judgment
• U.S.-based