diff --git a/content/manuals/ai/sandboxes/configuration/models.md b/content/manuals/ai/sandboxes/configuration/models.md index 74c7eecc89..2d77a643f0 100644 --- a/content/manuals/ai/sandboxes/configuration/models.md +++ b/content/manuals/ai/sandboxes/configuration/models.md @@ -23,7 +23,7 @@ Docker Sandboxes includes `llmman`, a model management tool installed alongside `sbx`. It serves local models and forwards requests to hosted providers or custom endpoints. You don't need to install it separately. -The Docker Sandboxes daemon starts `llmman` on your host and stops it when +By default, the Docker Sandboxes daemon starts `llmman` on your host and stops it when the daemon stops. At startup, `llmman` downloads its llama.cpp runtime if needed, so the first startup requires network access for that download. The service starts even if you haven't used `--model`. @@ -34,6 +34,33 @@ On Linux, `llmman` first tries Docker or Podman to run the inference server in a container. If neither is usable, it tries a downloaded binary, then `llama-server` on the host's `PATH`. +### Disable the bundled service + +To prevent the daemon from starting the bundled server and downloading its +inference runtime, disable `model.server.enabled` and restart the daemon: + +```console +$ sbx settings set model.server.enabled false +$ sbx daemon restart +``` + +If the daemon isn't running, the settings command starts it before saving the +override, so the bundled service can start once before the restart applies +the change. + +You can still [use Ollama](#use-ollama). Other provider selections, including +hosted providers and custom endpoints, need a running `llmman` server. +If no server is ready while the bundled service is disabled, `sbx run --model` +fails immediately with a hint to re-enable it. An already-ready server remains +usable. + +To re-enable the bundled service: + +```console +$ sbx settings set model.server.enabled true +$ sbx daemon restart +``` + ## Enable model selection Run these commands on your host: diff --git a/content/manuals/ai/sandboxes/configuration/settings.md b/content/manuals/ai/sandboxes/configuration/settings.md index 6d57bd5256..948e748019 100644 --- a/content/manuals/ai/sandboxes/configuration/settings.md +++ b/content/manuals/ai/sandboxes/configuration/settings.md @@ -434,6 +434,25 @@ timestamp from a timestamp authority instead of a transparency log timestamp. Leave this at `false` when your signing workflow uses the public transparency log. It has no effect on signatures verified with a public key. +### Model server + +#### model.server.enabled {.wrap-anywhere} + +{{< setting-metadata type="boolean" default="true" >}} + +Set this to `false` to prevent the daemon from starting the bundled `llmman` +model server and downloading its inference runtime: + +```console +$ sbx settings set model.server.enabled false +$ sbx daemon restart +``` + +Changes require a daemon restart. This setting has no environment variable +override. See [Disable the bundled service](models.md#disable-the-bundled-service) +for initial daemon startup behavior, provider requirements, and how to re-enable +the service. + ### MCP gateway #### mcp.forceLocalGateway {.wrap-anywhere}