AI Agent Evaluator
$25 to $45/hIndicative range provided by micro1
Posted on October 9, 2026 · Applications until December 8, 2026
I am sending you to the official micro1 page. Applying is free and in English.
I earn a commission from the platform when a candidate I referred gets hired. It changes nothing for you.
The AI Agent Evaluator tests everyday automated scenarios on a smartphone, approves key steps, and scores results. Requirements include a Bachelor degree, two to three years of professional experience, and an active personal AI account.
Description in English, as published by micro1.
Role Title: AI Agent Evaluator
Role Type: Contractor
Location: U.S based
Opportunity Snapshot
Pay: $25-45/hour
Time per task: ~1.5 hours per run (evidence and scoring included)
Project kickoff: Monday, October 12
Onboarding: Today (Fri), Saturday, or Sunday, so you’re ready to start Monday October 10th
You’ll need: An active personal popular AI account (Free, Go, or Plus; not Pro), an active personal and a smartphone that can screen record
Your tasks: Run everyday errands with two AI agents, approve key steps, screen record, and score the results
About the role
AI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries. We want to know how well they actually do it, from the point of view of a real user. That’s you.
You’ll test two AI agents side by side, using your own accounts on your phone, and tell us what worked, what didn’t, and how it felt.
What you’ll do
- Run real-life scenarios: bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms
- Approve or stop the agent at key steps
- Screen record every attempt
- Score each run on completion, quality, control issues, satisfaction, and time
- Share clear written feedback on the agent experience
Who we’re looking for
- US-based, with a Bachelor’s degree
- Someone who already uses AI in daily life
- 2-3 years of experience, preferably in engineering, business management, or office/operations
- Detail-oriented, great at following instructions and documenting each step
- Clear written communication
- Generalists welcome, domain experts preferred
- No coding or AI background needed
Why you’ll like it
- Get paid to use the AI tools you already use
- Work entirely from your phone, on your schedule
- Have a direct say in how next-generation AI agents get built and improved
How the selection process works
- Apply
- Complete the screening questions
- Interview: 15-30 minutes max
- Profile review: if everything looks good, we present you to the client
- Onboarding: our Talent Ops team will contact you to get you started
Required skills
- ChatGPT
- Muse
- AI agents
- Attention to Detail
- Following Instructions
- Screen Recording
50 openings
Only: United States
About micro1
micro1 is an AI data lab that hires remote experts to train and evaluate models. The original posting is on their site.
View this job on micro1Similar jobs
Everyday AI Agent Tester
This contractor evaluates two AI agents on a smartphone by running daily errands, approving key steps, screen recording, and scoring the results. Candidates need a Bachelor degree, two to three years of professional experience, and familiarity with AI tools.
- ChatGPT
- Muse
- AI agents
- +3
Posted yesterday$25 to $45/h
Gmail & Google Calendar AI Assistant Evaluator
This remote contractor position involves completing realistic digital tasks, such as emailing and scheduling, while evaluating and annotating AI assistant responses. Candidates must reside in the US, use Gmail and Google Calendar daily, and possess an iPhone with iMessage alongside a Facebook or Instagram account.
- Gmail
- Google Calendar
- AI Usage
- +3
Posted 15 days ago$15 to $30/h
Generalist Writer
The Generalist Writer evaluates AI generated text and written content to improve the accuracy, coherence, and overall quality of AI systems. Candidates must be native US English speakers with professional writing experience and availability for at least twenty hours per week.
- professional writing
- editing
- content review
- +5
Posted 59 days ago$40 to $50/h
Generalist Expert
This remote generalist role involves training and evaluating frontier AI models by writing prompts, grading outputs, and flagging reasoning errors. The position requires strong professional or academic reading skills and clear written communication of reasoning.
Posted 25 days ago$40 to $70/h