researchAI Infrastructure7d ago
SlackTier Optimization Reduces Memory Usage for LLM Serving via NVMe Offloading
Facts
Researchers have introduced SlackTier, a memory-efficient system for SGLang that offloads LLM n-gram lookup tables from host DRAM to NVMe storage. By utilizing per-layer prefetch deadlines, the system reduces host memory consumption from 212 GB to 34 GB while enabling significant increases in KV cache capacity for agent workloads.
- ▸SlackTier reduces host memory requirements for DeepSeek-V4.1 serving from 212 GiB to 34 GiB.
- ▸The system uses NVMe storage for n-gram tables with a maximum output token latency increase of 1.4%.
- ▸Offloading memory to host KV cache improves median throughput for agent workloads by 36%.
- ▸Median time to first token for agent workloads was reduced from 38 seconds to 24 seconds.
- ▸SlackTier integration into SGLang operates within the decode CUDA graph to maintain bit-exact results.
- Occurred
- 2026-11-10 00:47 UTC
- First reported
- 2026-11-10 00:47 UTC
- Last updated
- 2026-11-11 02:05 UTC
- Event type
- research
- Region
- Global
Entities
Categories
Source timeline · 1
- arXivFirst-party2026-11-10 00:47
Community vote
Sign in to vote on importance and sentiment.
Platform Signals
Impact score
68
AI sentiment
PositiveEvent significance30%1
Entity importance20%1
Source credibility20%1
Novelty15%1
Corroboration15%1
Confidence1
Novelty1
Corroboration1
Community
ImportanceNo votes
Strategic sentiment
Positive 0%Neutral 0%Negative 0%
Platform signals are algorithmic and community-derived indicators. They are not investment recommendations and do not predict valuation or performance.
Related events
No events yet.