Proximal Maps Model Failures in Engineering

Publishing a hard public benchmark turns Proximal from a black box vendor into a mapmaker for where coding agents still break. FrontierSWE is built around 17 ultra long horizon tasks across implementation, performance engineering, and ML research, with up to 20 hours per run, and Proximal reports that most frontier models barely make progress on many tasks. That gives buyers a concrete view of which jobs still need better data, better training, or tighter human supervision.1 2 3 4

• The benchmark is aimed at work that looks more like real senior engineering than benchmark puzzle solving. Tasks include optimizing a compiler, building a PostgreSQL compatible server on SQLite, and improving ML optimizers. That makes weak scores useful sales evidence for customers buying help on hard technical workflows.1 2
• This also supports Proximal's product loop. The company says it trains models on its own RL infrastructure and uses post training to test whether its data actually improves performance. A public benchmark lets it show that loop on a problem set aligned with the kind of data products it sells.5 6 7
• Open benchmarks have become a distribution channel in AI infrastructure. Other companies in evals and data, including AfterQuery, Vals, Arena, and Design Arena, use benchmarks or eval systems to prove quality, collect demand, or shape model buying decisions. Proximal is applying that playbook specifically to hard coding and research tasks.8 9 10 11

The next step is a tighter link between benchmark visibility and paid improvement work. As model labs and enterprises chase longer running software agents, the winners will be the companies that can both show exactly where models fail and supply the data, environments, and training loops that move those scores upward on real engineering tasks.1 5 6 11