Behind every frontier model is a quiet workforce of people who already have a day job. Radiologists grade chatbot answers between shifts. Attorneys write the reference opinion a model is measured against. Physicists try to break a system before the public can. This post explains what that work is, what public reporting says it pays, why labs now want credentials rather than crowds, and what the job asks of the people who do it well.
What the work actually is.
The phrase “training data” hides a dozen distinct jobs. Most expert programs combine several of them.
| Task | What you do | Who does it well |
|---|---|---|
| Preference ranking | Read two or more model responses to the same prompt and say which is better, and why. This is the raw material of reinforcement learning from human feedback (RLHF) [5]. | People who can hold a stable standard across hundreds of comparisons without drifting. |
| Rubric grading | Score a response against written criteria: did it include the warning, cite the rule, flag the uncertainty. HealthBench is built from 48,562 physician-written criteria across 5,000 conversations [8]. | Clinicians, lawyers, and editors used to checklists and audits. |
| Reference answers | Write the deliverable a model should have produced: a diagnosis, a memo, a valuation, a proof. GDPval tasks came from professionals averaging 14 years of experience [9]. | Practitioners who write for a living and can show their reasoning. |
| Evaluation design | Author hard, unambiguous questions that cannot be looked up. Humanity’s Last Exam has 2,500 of them, written by subject-matter experts worldwide [10]. | Researchers and teachers who know where a field’s real difficulty lies. |
| Red teaming | Probe a model for failures in your own domain: harmful medical advice, bad legal guidance, biosecurity uplift [11, 12]. | Specialists with an adversarial habit of mind and a tolerance for tedium. |
| Code review datasets | Judge pull requests, rank candidate fixes, write tests that a correct patch must pass. | Senior engineers who review more than they write. |
The common thread is judgment. The model already produces fluent text. What it cannot produce is a verdict on whether that text is right, and that is what labs are buying.
Why labs pay for credentials now.
Three findings explain the shift from crowd annotation to credentialed experts.
First, feedback quality matters more than model size. In the original InstructGPT work, about 40 contractors hired through Upwork and a labeling vendor produced the comparison data, and the resulting 1.3-billion-parameter model was preferred over the 175-billion-parameter GPT-3 it was built from [5]. A small amount of careful human preference did what a hundredfold increase in parameters could not.
Second, data quality can substitute for data quantity. “Textbooks Are All You Need” trained a 1.3-billion-parameter code model on roughly 7 billion tokens of curated, textbook-quality material and reached 50.6% pass@1 on HumanEval, a result that previously required far larger models trained on far more web data [6]. The lesson labs took from it was that a well-written explanation is worth more than a pile of scraped text.
Third, the easy work is moving to machines, which leaves the hard work to people. Google researchers showed that reinforcement learning from AI feedback (RLAIF) achieves performance comparable to RLHF on summarization and dialogue [7]. If a model can rank ordinary chat responses about as well as a crowd worker, the crowd worker’s marginal value falls. What a model cannot yet do reliably is tell whether a drug interaction warning is complete or whether a contract clause is enforceable in Delaware. That requires a person who would be liable for the answer in real life.
The public record shows labs acting on this. OpenAI opened a Red Teaming Network in 2023 seeking experts in healthcare, law, psychology, linguistics, finance, and other fields [11]. Anthropic describes domain-specific expert red teaming, including policy vulnerability testing with outside specialists, as a core part of its evaluation practice [12]. HealthBench recruited 262 physicians from 60 countries and 26 specialties to write its grading rubrics [8]. GDPval had three expert graders per task spend, on average, more than an hour on each blind comparison [9].
TIME’s account of the industry puts it plainly: the first era paid contractors “pennies per task” to label images; the current era seeks expert data written by people with PhDs who document their problem-solving process [2]. By mid-2025, TIME reported, each of the leading labs was spending roughly $1 billion a year on human data [2].
What it pays, according to public reporting.
The figures below are reported industry figures. They are not Mikra’s rates. Mikra’s openings carry their own rate bands, listed on each role.
- Bloomberg, in October 2026, quoted recruiter listings from one data vendor seeking software engineers at up to $120 an hour, condensed-matter physicists at $170 an hour, and civil rights attorneys at $400 an hour. A second platform told Bloomberg its average pay is $110 an hour [1]. Bloomberg was careful to note these come from listings and company statements, not payroll data.
- TIME, in October 2025, reported that the same platform advertised an average of $81 an hour and more than $200 an hour for “senior domain experts” with at least several years of professional experience [3].
- GDPval’s authors estimated the cost of expert grading at about $86 per comparison, derived from median occupational wages multiplied by observed grading time [9].
- At the other end of the market, Google’s “super raters,” many with advanced degrees, were reported in September 2025 to earn $28 to $32 an hour when hired directly by the contractor and $18 to $22 through third-party agencies for identical work [16].
- In 2023, TIME reported that Kenyan workers labeling toxic content for OpenAI, through an outsourcing firm, took home between $1.32 and $2 an hour, while the firm was paid $12.50 an hour per worker [13].
The spread is the story. The same label, “AI training,” covers a $2 task and a $400 one. What separates them is whether the lab needs a credential it cannot substitute.
Scale of the market: one vendor reported a $450 million annualized run rate in September 2025 and was raising at a valuation above $10 billion, with its CEO noting the run rate includes the portion paid out to contractors [4]. Bloomberg reported the same company discussing funding at $20 billion a year later [1]. These are vendor figures, not independent audits, and they measure gross flows rather than expert earnings.
What good looks like.
Labs do not pay for opinions. They pay for calibrated judgment that another expert would reproduce. Three habits distinguish people who last in this work.
Calibration. In InstructGPT, trained labelers agreed with one another 72.6% of the time, and held-out labelers 77.3%; the researchers themselves agreed at about 73% in earlier work [5]. GDPval’s expert graders agreed 71% of the time [9]. Those numbers are the ceiling, not a failure. A grader who agrees with the consensus far less than that is usually applying a private standard. A grader who agrees far more is usually not reading closely. Good evaluators know their own tendencies, whether they are harsh on tone or lenient on omissions, and correct for them.
Rubric discipline. A rubric exists so that two strangers reach the same score. That only works if the grader scores what the criterion says, not what they wish it said. HealthBench’s authors added consensus criteria only where a majority of physicians agreed, and wrote example-specific criteria individually [8]. The discipline is to grade against the text in front of you, record a disagreement with the rubric separately, and never silently reinterpret.
Honesty about uncertainty. The most valuable label is often “I am not sure, and here is why.” A physician who flags that a dosing answer depends on renal function the prompt did not supply is producing better data than one who guesses. Models learn from confident wrong labels exactly as readily as from confident right ones.
Writing matters more than people expect. A ranking without a rationale is a single bit. A ranking with two precise sentences explaining the error is something a reviewer can check and a model can learn from.
The parts people don’t talk about.
The industry’s record is uneven, and a reader deciding whether to do this work should know it.
Classification disputes are common. The U.S. Department of Labor investigated one large vendor for Fair Labor Standards Act compliance and dropped the inquiry in May 2025 without a stated reason; two former-worker lawsuits alleging underpayment and misclassification had been filed in the months before [14]. Another vendor was sued in a proposed class action in May 2025 by an annotator who alleged unpaid training and “near-impossible time limits for tasks that caused their pay to be docked” [15]. The allegations are unproven, but the pattern they describe, unpaid onboarding and per-task clocks that quietly cut rates, is one experts should ask about before accepting work.
Pay transparency is weak. The Google rater case showed a $10-an-hour gap between two people doing the same task through different agencies, and two workers filed complaints with the National Labor Relations Board after they said they were dismissed for raising it [16]. More than 200 raters were let go with little warning in two rounds [16].
Data about the workers themselves can leak. In March 2026 a supply-chain attack on an open-source library gave attackers access to one major vendor’s systems; hackers claimed 4 terabytes including candidate profiles and personal information, five contractors sued, and at least one lab paused its contracts [17]. Anyone uploading a passport and a video interview to a platform should ask how that material is stored and for how long.
The harshest work remains poorly paid. The content that makes models refuse to describe abuse was labeled, in 2023, by people earning under $2 an hour reading descriptions of it for a nine-hour shift [13]. Expert rates at the top of the market have not changed that.
And the expert tier is small. A labor sociologist told Bloomberg that highly paid specialists participate in relatively few projects compared with low-paid generalists, and called them “the cherry on top” of the data workforce [1]. The $400-an-hour listing is real, but it is not the median experience of someone doing AI data work.
Non-disclosure agreements are standard across the industry. They are reasonable for unreleased models. They become a problem when they prevent a worker from discussing pay or conditions, and some of the labor complaints above arose in exactly that gap.
How Mikra runs it.
Mikra hires practitioners to write and check the tasks in its own environments. Each role is a specific, named opening rather than a general pool, so you know the domain, the task type, the hours, and the rate band before you apply. Every applicant passes identity and credential verification. The calibration step is paid, and the work from it is reviewed by a senior expert in the same field who sends written feedback, because the agreement rates above are reached by practice, not by instinct. Payment follows a fixed schedule set out in the engagement, and the rate on the listing is the rate you are paid. The current openings are at /jobs.
Intelligence has a source. For frontier models, that source is increasingly someone who spent a decade learning a field and is now willing to explain, carefully and repeatedly, what right looks like.