Session V · 4 February 2026

Exploring Docker Model Runner

Presented by Julian Silveira

Running and testing models locally with Docker's own tooling, and when to pick it over Ollama or vLLM.

What was covered

Two parts. A distribution side, OCI-compatible, that packages model weights and metadata, and a runtime engine.
Familiar interfaces. It exposes an OpenAI-compatible API and plugs into Docker Compose, so changing a model means changing the Compose definition.
A catalogue like Docker Hub. You can package your own models and publish them as images.
How it compares. Ollama suits individual developers and personal projects. Docker Model Runner suits teams that want a standard runtime through Compose. vLLM suits high-throughput production.

Data constraints. Useful when data rules out cloud providers and the project needs local models, for example for an MVP or proof of concept.
Support. Ollama supports more models, while Docker Model Runner is more standardised. On a single laptop the speed difference may be small.
Maturity. One of us found it easier to use than Ollama but still beta-like, for instance when setting the context window from the UI.
Integration. The main draw is adding a model to an existing application through a YAML file.
Experience. Earlier tests with Ollama had been slow and unstable on one machine, and the room wanted to try this one.

For a team that needs the same local-model setup on every machine, this is the option to try first.