open source releaseAI Agent Evaluation5d ago
Release of drydock-eval Evaluation Kit for Small Coding Agents
Facts
Researchers released drydock-eval, an open-source evaluation kit designed to benchmark small coding agents like Gemma 4 E4B. The kit provides containerized, local assessment tools for performance and security, including a full dataset of over 9,000 experimental runs.
- ▸drydock-eval supports evaluation speeds of 250 runs per hour on a single RTX 5090.
- ▸The framework includes 200 scripted escape tests for security and provides token-level comparison metrics.
- ▸The full dataset includes 9,660 rows of agent runs, 2,494 grading controls, and 72 serving measurements.
- ▸Performance testing on 31B and E4B models showed specific token decoding speeds on L4 GPUs.
- ▸No significant solve-rate change was detected across four different prompt divergences in a pre-registered experiment.
- Occurred
- 2026-11-12 08:54 UTC
- First reported
- 2026-11-12 08:54 UTC
- Last updated
- 2026-11-13 06:20 UTC
- Event type
- open source release
- Region
- global
Entities
Categories
Source timeline · 1
- arXivFirst-party2026-11-12 08:54
Community vote
Sign in to vote on importance and sentiment.
Platform Signals
Impact score
70
AI sentiment
NeutralEvent significance30%1
Entity importance20%1
Source credibility20%1
Novelty15%1
Corroboration15%1
Confidence1
Novelty1
Corroboration1
Community
ImportanceNo votes
Strategic sentiment
Positive 0%Neutral 0%Negative 0%
Platform signals are algorithmic and community-derived indicators. They are not investment recommendations and do not predict valuation or performance.