From Scale to Specialist Infrastructure

This shift turns human data from a scale game into a quality and workflow game. Early LLM training needed armies of low cost raters to rank outputs and flag bad answers. Post training now leans more on doctors, lawyers, scientists, coders, linguists, and culturally specific evaluators who can teach or test a model on narrow tasks, while automation handles more of the repetitive labeling work.

• Scale’s original advantage was bundling software with a large contractor base paid per task through Remotasks. That works well for image boxes and broad RLHF, but it is less defensible when buyers want smaller expert cohorts and only pay for high judgment tasks instead of bulk annotation volume.
  1 sacra 3 sacra
• Mercor, Handshake, Office Hours, and Prolific represent the new shape of the market. They win by finding and verifying specific people, a physician for healthcare evals, a PhD physicist for reasoning tasks, or a fluent local user for cultural nuance, then plugging them into API or managed workflows.
  2 sacra 3 sacra 4 sacra
• The xAI cuts matter because they show labor can be re segmented very quickly. A team built for general annotation can shrink fast once models and tooling automate the easy work, leaving a smaller layer of higher paid specialists who review edge cases, safety issues, and domain specific outputs.
  1 sacra 2 sacra 4 sacra

Going forward, the winners in human data will look less like classic BPOs and more like expert infrastructure. The durable platforms will combine software, profiling, QA, and deep pools of verified specialists, so they can route each task to the cheapest acceptable machine step and reserve humans for the last mile where judgment still matters most.