Automating QC and Ops with LLMs
Ops lead at Scale AI on using Claude Cowork & Codex for QC automation and multi-tool debugging at scale
This shows that agent tools are starting to replace a layer of routine operations analysis, not just software work. At Scale AI, people are no longer writing many dashboard queries or taxonomy edits by hand. They describe the business change they want, then an agent writes the SQL, finds the right workflow step, proposes the exact edit, and asks for approval before pushing it live. This turns code into hidden plumbing for ops work.
-
The reporting example is concrete. Teams use Redash queries with SQL underneath, and pick Gemini, Claude Code, or Codex based on task complexity. The output is not an app for engineers. It is an internal dashboard or report that operations teams use to track QMS, QC, and quality scores.
-
The taxonomy workflow shows why this matters. Instead of manually hunting through a multi step content flow, the agent inspects the taxonomy, identifies the right insertion point, shows a before and after text plan, opens a live preview, and only implements after approval. The reported accuracy over the prior two to three months was above 90%.
-
This fits a broader pattern in agent adoption. Single tool and two to three tool jobs are now dependable enough to hand off, while four or five connected tools still break too often. That makes dashboard generation and guided workflow edits attractive early use cases, because they are valuable but still bounded.
The next step is ops software that speaks in business terms while hiding GitHub, SQL, and API details. As those interfaces improve, more of the analyst and operations manager workflow will become approve or revise, rather than build from scratch. The winning products will make complex back end actions feel like editing a document, not running a dev tool.