Gross earnings for a part-time prompt evaluator| Hours a week | At $18.16 an hour | At $25 an hour | At $30 an hour |
|---|
| 5 hours | $91 | $125 | $150 |
| 10 hours | $182 | $250 | $300 |
| 20 hours | $363 | $500 | $600 |
| 10 hours a week over 52 weeks | $9,419 | $13,000 | $15,600 |
Why does the specialist band pay more?
Because a client is buying agreement between evaluators, and that only holds with trained judgement.
Three factors drive the premium. The error cost is higher, because a wrong rating corrupts the dataset the client is training on. The qualification is narrower, because a rubric on medical or legal content needs someone who understands the domain.
The third is the training. A new evaluator is calibrated against existing raters, which is unpaid work that a client covers by paying the specialist rate.
Our guide to part time remote data annotation collector covers the labelling side of the same work, where the rate is lower and the tasks are faster.
How is the work paid?
Per rating, per hour, or per guideline, and the first one makes the volume the wage.
Per rating is the marketplace model. It looks generous per item and pays least per hour, because the time between items is unpaid and the queue moves in bursts.
Per hour is the employee model, and it arrives with benefits worth 30.1 percent of the employer hourly cost, which is why the hourly rate looks lower than the per item headline.
Per guideline is the contract model, and it pays best because the client is buying a defined piece of work rather than an hour of effort.
How the three structures compare| Structure | Rate basis | Main risk |
|---|
| Per rating | Completed items | Queue gaps between batches |
| Per hour, employee | Logged time | Volume of work offered |
| Per guideline or project | Delivered work | Unpaid research and revision |
What should you check before you accept?
Four questions, and the first one decides whether the work is worth doing.
The first is whether the calibration training is paid, because uncalibrated ratings are worthless to the client. The second is how the agreement rate is measured, since that is what the client pays for and what a good evaluator protects.
The third is the domain, because a rubric on clinical or legal content is a different job from rating general chat responses. The fourth is the turnaround, since a deadline on quality work is how errors get made.
Our guide to part time remote search engine evaluator covers the same four questions on relevance judging.
How do you find evaluation work?
Three channels, and the third one carries the specialist rate.
Platforms are the entry point, where the volume work and the lower rate both sit. Vendors and data annotation firms come next, with steadier hours and a calibration path.
The third is the research team or the model developer, which hires evaluators for a defined project. Those roles usually ask for a sample exercise, which is also the fastest way to prove the judgement.
Our guide to part time remote chat support jobs covers the service work that runs alongside it.
Two questions to ask. What is the calibration process and how long it takes. And how the agreement rate is measured and what happens if it falls.
What changes about evaluation work next?
Expect the work to move up the value chain, because basic rating is exactly what automation does well.
Three forces shape the next two years. Models are getting better at the routine rubric work, which pushes human effort towards writing the rubrics and handling the edge cases. The volume of evaluation work keeps growing as models ship more often and in more languages.
The third is the regulation of model output, which creates a demand for documented human judgement in the areas where the output has to be defensible.
For a candidate the conclusion is that the task is becoming the rubric rather than the rating, and the rate follows the part that needs a domain qualification.
Next steps
Look for the roles that write the guidelines, because that is where the specialist rate sits.
- Compare the rate with the $18.17 median for data entry keyers
- Ask whether the calibration training is paid and how long it takes
- Ask how the agreement rate is measured and what happens if it falls
- Check the domain, because clinical and legal content pay differently
- Confirm whether you are paid per rating, per hour or per guideline
- Keep a log of items and time, because the rate only makes sense per item
One final thought. The pay figures here are occupation medians and published platform rates, which is the best guide available before you apply.
The Bureau of Labor Statistics occupation wage estimates give the percentile ladder behind these numbers, and the Federal Trade Commission job scam guidance covers the warning signs that rule out the fake listings.
Photo: ornanvelazquez, Pixabay Content License, via Pixabay (https://pixabay.com/photos/sony-a7-sony-a7-camera-sony-camera-7517301/)
Fazlur Rahman
