Run a Private vLLM Server on Hugging Face Jobs with One Command

Spinning up a private, OpenAI-compatible LLM endpoint typically involves provisioning servers, configuring Kubernetes, or relying on managed services that may be overkill for quick experiments. Hugging Face Jobs exposes this tension: how do you get a model serving in minutes without the operational overhead, while still keeping it private and pay-per-second? The article shows that HF Jobs bridges this gap by offering a docker run-like experience on Hugging Face infrastructure, where you don’t need to manage any infrastructure yourself.

The technical path is strikingly simple: a single hf jobs run command with the --expose 8000 flag launches a vLLM server from the official vllm/vllm-openai image on a chosen GPU flavor like a10g-large. The command prints a publicly reachable but gated URL that requires an HF token with read access to the job’s namespace. You query it with the OpenAI client or curl, passing the token as the bearer token. The entire flow — from image selection to paying per second — replaces what would be days of DevOps toil. The article also extends this to larger models with --tensor-parallel-size, SSH debugging with --ssh, and tool-calling for agent backends with --enable-auto-tool-choice.

For a serious builder, the takeaway is that HF Jobs is the fastest path to a private, self-hosted LLM endpoint for non-production workloads like evals, batch generation, or prototyping. It’s not a substitute for Inference Endpoints when you need scale-to-zero or fine-grained access control, but it eliminates friction for ephemeral serving. The article also reinforces a practical debugging pattern: if a large model fails to start, dialing down --max-model-len and --max-num-seqs is the first thing to try, especially with hybrid architectures like Qwen3.5-122B.