benchmarkAI Benchmarking6d ago
Researchers Introduce WordInferBench to Evaluate AI Agent Contextual Inference in Document Editing
Facts
WordInferBench is a new benchmark designed to assess how effectively AI coding agents can infer formatting attributes from document context rather than relying on explicit instructions. Testing of six current coding agents reveals a significant performance drop when tasks require contextual inference compared to descriptive instructions, with Claude Code reaching only 40.7% on inferential tasks.
- ▸WordInferBench comprises 280 document-editing instances derived from Common Crawl.
- ▸The benchmark includes three instruction conditions: descriptive, referential, and inferential.
- ▸The test set covers 13 distinct object types and 81 unique document attributes.
- ▸Six coding-agent systems were evaluated using the new benchmark framework.
- ▸Claude Code with Opus 4.6 demonstrated a 62.9% pass rate on descriptive tasks but only 40.7% on inferential tasks.
- Occurred
- 2026-11-11 08:36 UTC
- First reported
- 2026-11-11 08:36 UTC
- Last updated
- 2026-11-13 04:15 UTC
- Event type
- benchmark
- Region
- Global
Entities
Categories
Source timeline · 1
- arXivFirst-party2026-11-11 08:36
Community vote
Sign in to vote on importance and sentiment.
Platform Signals
Impact score
68
AI sentiment
NeutralEvent significance30%1
Entity importance20%1
Source credibility20%1
Novelty15%1
Corroboration15%1
Confidence1
Novelty1
Corroboration1
Community
ImportanceNo votes
Strategic sentiment
Positive 0%Neutral 0%Negative 0%
Platform signals are algorithmic and community-derived indicators. They are not investment recommendations and do not predict valuation or performance.
Related events
- 1dIntroduction of MASTA Multi-Agent System for Security Protocol Formal Verification
- 5dFolio: A New Context Management System for Long-Horizon AI Agents
- 5dIntroduction of Singularity-Bench for Evaluating Autonomous AI Development
- 6dIntroduction of RSI-PDEbench and RSI-PDE Agent Framework for Scientific Computing
- 6dResend Discusses Designing APIs for AI Agent Consumption
- 7dStudy Evaluates Claude Code as an Agentic Grading System for Handwritten Exams