The article presents a constraint-aware GPU allocator built by the author’s team, benchmarked against a FIFO scheduler across seven scenarios.

On identical hardware and workloads, GPU utilization rose by as much as 33 percentage points, and priority-weighted output increased by up to 105%, with an average gain of 52%.

The key insight is that ordering allocation decisions—rather than hardware changes—drives the improvement.

The core problem is managing two incompatible allocation shapes: batch-like jobs (training, batch inference, quantization) that require contiguous, uninterrupted GPU blocks, and real-time inference that is elastic and demands per-timestep capacity.

FIFO scheduling wastes capacity through two mechanisms: static reservations for peak real-time demand (holding idle GPUs all day) and arrival-order placement that ignores job priority and future fit.

The allocator treats real-time demand as a time-varying curve, allocating GPUs per timestep and reclaiming idle capacity for batch work during troughs, bounded by a swap cost limit.

Batch jobs are placed by priority across a rolling 24-hour horizon rather than arrival order.

The formal model defines five constraints (one GPU per job per timestep, demand ranges, contiguous power-of-two blocks for batch, swap cap for real-time, no preemption) and a dual-objective function: reward for allocating batch jobs (priority × time-decay weight) and penalty for unmet real-time demand (5–10× higher weight than allocation).

The relative weights encode service-level policy as a single number.

Because solving this NP-hard problem per request must be fast, a heuristic is designed to satisfy the formal model’s constraints by construction, running in 1–2 ms on contended scenarios and 15 ms at 64 GPUs with 30 jobs.

The heuristic sees all queued jobs before placing any, enabling it to hold shapes that fit remaining work. A uniform-priority test (all jobs identical priority) still yields 23.

1% more value, proving horizon-aware planning alone contributes.

Performance relies on accurate demand forecasts: separate estimators for training (22 features including 10 concrete training variants), quantization (per algorithm with safety margins), and real-time inference (weekly profile from hourly history, mapped via swap cost).

To handle forecast error, the scheduler optimizes a 24-hour horizon but commits only one timestep, re-running every 30–60 minutes, so errors are absorbed by re-optimization.

The article concludes that structural discipline—encoding physical constraints into ordering—achieves more than algorithmic sophistication, analogous to how airlines improved utilization via turnaround sequencing and crew rostering.