docker/docs
1
0
Fork 0
mirror of https://github.com/docker/docs.git synced 2026-10-28 07:15:09 +03:00
Code Activity

docs: document bundled model server opt-out

Browse Source 
The model configuration pages omitted the bundled server startup setting.
Document disabling and re-enabling it, restart timing, and provider requirements.
This commit is contained in:
David Karlsson
2026-10-23 13:27:08 +00:00
parent b71dd84bd2
commit 5227b9ee7f
2 changed files with 47 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
content/manuals/ai/sandboxes/configuration/models.md
View File

@@ -23,7 +23,7 @@ Docker Sandboxes includes `llmman`, a model management tool installed alongside
`sbx`. It serves local models and forwards requests to hosted providers or
custom endpoints. You don't need to install it separately.
The Docker Sandboxes daemon starts `llmman` on your host and stops it when
By default, the Docker Sandboxes daemon starts `llmman` on your host and stops it when
the daemon stops. At startup, `llmman` downloads its llama.cpp runtime if
needed, so the first startup requires network access for that download. The service starts even if you haven't used `--model`.
@@ -34,6 +34,33 @@ On Linux, `llmman` first tries Docker or Podman to run the inference server
in a container. If neither is usable, it tries a downloaded binary, then
`llama-server` on the host's `PATH`.
### Disable the bundled service
To prevent the daemon from starting the bundled server and downloading its
inference runtime, disable `model.server.enabled` and restart the daemon:
```console
$ sbx settings set model.server.enabled false
$ sbx daemon restart
```
If the daemon isn't running, the settings command starts it before saving the
override, so the bundled service can start once before the restart applies
the change.
You can still [use Ollama](#use-ollama). Other provider selections, including
hosted providers and custom endpoints, need a running `llmman` server.
If no server is ready while the bundled service is disabled, `sbx run --model`
fails immediately with a hint to re-enable it. An already-ready server remains
usable.
To re-enable the bundled service:
```console
$ sbx settings set model.server.enabled true
$ sbx daemon restart
```
## Enable model selection
Run these commands on your host:

19
content/manuals/ai/sandboxes/configuration/settings.md
View File

@@ -434,6 +434,25 @@ timestamp from a timestamp authority instead of a transparency log timestamp.
Leave this at `false` when your signing workflow uses the public transparency
log. It has no effect on signatures verified with a public key.
### Model server
#### model.server.enabled {.wrap-anywhere}
{{< setting-metadata type="boolean" default="true" >}}
Set this to `false` to prevent the daemon from starting the bundled `llmman`
model server and downloading its inference runtime:
```console
$ sbx settings set model.server.enabled false
$ sbx daemon restart
```
Changes require a daemon restart. This setting has no environment variable
override. See [Disable the bundled service](models.md#disable-the-bundled-service)
for initial daemon startup behavior, provider requirements, and how to re-enable
the service.
### MCP gateway
#### mcp.forceLocalGateway {.wrap-anywhere}