Databricks hosted the inaugural Grounded Reasoning Cup, a live competition testing AI agents on complex questions over large, enterprise-style document collections. Eleven academic teams competed using OfficeQA and a new OfficeQA Pro V2 benchmark. Teams had two months to build agents with models from partner labs OpenAI, Anthropic, and Google DeepMind, then answered timed questions over newly released U.S. Treasury financial reports. Stanford won with 63.3% accuracy, clearly outperforming offline baselines. Results showed strong gains from document preprocessing, targeted retrieval, parallel agents, structured tool use, and verification, yet nearly one-fifth of questions remained unsolved. Organizers urge continued research using the public OfficeQA benchmarks to improve enterprise grounded reasoning.
This update represents a notable development in the Ai sector. Organizations and founders tracking this space should evaluate potential strategic and technical implications on their operations.