Feed
benchmarkAI Benchmarking6d ago

Researchers Introduce WordInferBench to Evaluate AI Agent Contextual Inference in Document Editing

Facts

WordInferBench is a new benchmark designed to assess how effectively AI coding agents can infer formatting attributes from document context rather than relying on explicit instructions. Testing of six current coding agents reveals a significant performance drop when tasks require contextual inference compared to descriptive instructions, with Claude Code reaching only 40.7% on inferential tasks.

  • ▸WordInferBench comprises 280 document-editing instances derived from Common Crawl.
  • ▸The benchmark includes three instruction conditions: descriptive, referential, and inferential.
  • ▸The test set covers 13 distinct object types and 81 unique document attributes.
  • ▸Six coding-agent systems were evaluated using the new benchmark framework.
  • ▸Claude Code with Opus 4.6 demonstrated a 62.9% pass rate on descriptive tasks but only 40.7% on inferential tasks.
Occurred
2026-11-11 08:36 UTC
First reported
2026-11-11 08:36 UTC
Last updated
2026-11-13 04:15 UTC
Event type
benchmark
Region
Global

Entities

Categories

Source timeline · 1

  • arXivFirst-party2026-11-11 08:36
Community vote

Sign in to vote on importance and sentiment.

Platform Signals

Impact score
68
AI sentiment
Neutral
Event significance30%1
Entity importance20%1
Source credibility20%1
Novelty15%1
Corroboration15%1
Confidence1
Novelty1
Corroboration1
Community
ImportanceNo votes
Strategic sentiment
Positive 0%Neutral 0%Negative 0%

Platform signals are algorithmic and community-derived indicators. They are not investment recommendations and do not predict valuation or performance.