From Bulk Labeling To Expert Evaluation

The important shift is that labeling is moving from a high volume outsourced factory job into a smaller, more specialized step inside the product workflow. Earlier demand was driven by open ended computer vision problems like autonomous driving, where every new weather, lighting, and road situation created more edge cases to label. As pre trained models get better, many business use cases can be tuned with tens or hundreds of examples, often labeled by the product team itself, not a large external workforce.

• Autonomous driving was the clearest case for endless labeling demand because the real world keeps changing. That market has narrowed. Cruise was shut down by GM in December 2024, while California autonomous vehicle test miles fell 50% in 2024 as the field consolidated around fewer players like Waymo.
  1 sacra 4 apnews 5 techcrunch
• Few shot and transfer learning change the economics because the bottleneck becomes picking the right examples, not buying thousands of labels. In Nyckel's workflow, customers can upload around 100 examples, label them in about 20 minutes, train in seconds, and put a model into production without building a full MLOps stack.
  1 sacra
• Human work is not disappearing, it is moving up the value chain. More recent demand is shifting toward expert evaluation, red teaming, and validation, where labs need smaller groups with domain knowledge or cultural context. That favors vendors built around quality and expertise over bulk annotation volume alone.
  2 sacra 3 sacra

Going forward, the winners are likely to be companies that wrap labeling, evaluation, and model improvement into one workflow. Pure pay per label businesses should keep losing share in mainstream use cases, while higher value services around expert judgment, safety testing, and model validation become the durable part of the market.