BLACKWELL INFERENCE
Auto routes each new conversation to the least-loaded warm model, then keeps it pinned for cache locality.
Use AutoWaiting for the inference backends.
dgx.cautela.io/v1
MODEL POOL
Waiting for backend telemetry…
GATEWAY
REAL TIME
Waiting for lane telemetry…
ANONYMIZED STREAM
| when | lane | model | in tok | cached | out tok | first token | decode tok/s | total | outcome |
|---|---|---|---|---|---|---|---|---|---|
| Waiting for requests… | |||||||||
LAST 256 REQUESTS
Content-free timing and token aggregates from the gateway response path.
Waiting for measured requests…
ACCELERATORS
Utilization, board power, and hottest observed GPU temperature.
Waiting for GPU telemetry…
LIFETIME
INTRA-DGX FABRIC
Waiting for per-link counters…
AUTO ROUTING
model: auto sends a new conversation to the least-loaded warm model and keeps future turns pinned there. Ask the operator for a personal dgx_… key and store it in an environment variable.
Route on loadChoose available capacity when a conversation begins.
Stay pinnedPreserve model behavior and prefix-cache locality across turns.
Override anytimeSelect a physical model above when the workload requires it.
Claude Code talks directly to the Anthropic-compatible Messages endpoint.
Loading routing mode…An in-memory provider config points OpenCode at the OpenAI-compatible endpoint.
Loading routing mode…Use a stable affinity value for exact per-agent pinning.
Loading routing mode…