Feed
open source releaseAI Agent Evaluation5d ago

Release of drydock-eval Evaluation Kit for Small Coding Agents

Facts

Researchers released drydock-eval, an open-source evaluation kit designed to benchmark small coding agents like Gemma 4 E4B. The kit provides containerized, local assessment tools for performance and security, including a full dataset of over 9,000 experimental runs.

  • ▸drydock-eval supports evaluation speeds of 250 runs per hour on a single RTX 5090.
  • ▸The framework includes 200 scripted escape tests for security and provides token-level comparison metrics.
  • ▸The full dataset includes 9,660 rows of agent runs, 2,494 grading controls, and 72 serving measurements.
  • ▸Performance testing on 31B and E4B models showed specific token decoding speeds on L4 GPUs.
  • ▸No significant solve-rate change was detected across four different prompt divergences in a pre-registered experiment.
Occurred
2026-11-12 08:54 UTC
First reported
2026-11-12 08:54 UTC
Last updated
2026-11-13 06:20 UTC
Event type
open source release
Region
global

Entities

Categories

Source timeline · 1

  • arXivFirst-party2026-11-12 08:54
Community vote

Sign in to vote on importance and sentiment.

Platform Signals

Impact score
70
AI sentiment
Neutral
Event significance30%1
Entity importance20%1
Source credibility20%1
Novelty15%1
Corroboration15%1
Confidence1
Novelty1
Corroboration1
Community
ImportanceNo votes
Strategic sentiment
Positive 0%Neutral 0%Negative 0%

Platform signals are algorithmic and community-derived indicators. They are not investment recommendations and do not predict valuation or performance.