Model hosting with /llmhost, GPU rentals, and /restart for live sessions
I am really excited to have /llmhost in this release. A lot of developers already know how to set up and serve their own LLMs, and the goal here is to give you more independence and control over your workflow by integrating that setup directly into Mercury. Instead of managing servers, tunnels and model configs separately by hand, you can open /llmhost, pick the model you want to work with first, and see where it can run right now: on a pod you already have running, on your own computer (mercury host), or on a GPU rented from Runpod, Vast.ai or Lambda.
Each model page in /llmhost shows the quantization options, how the GPU memory budget splits across parallel conversations (conversations × context), and the default thinking level. Before confirming a rental, or in place on a running llama.cpp Runpod pod, Tune adjusts quantization, context splits and default thinking, with automatic rollback if an in-place reload fails. On Runpod, you can also keep revision-verified model weights on a fast-load network or global volume so warm startups verify the mounted files by seeded sampling in seconds instead of downloading weights again, and 19 trillion-parameter recipes (Kimi K3, Qwen3.8-2.4T-A95B, DeepSeek V4 Pro, MiMo-V2.6-Pro-RL, Mistral Large 3 675B and GLM-5.3) support single-node pods and multi-node Runpod Instant Clusters with hourly compute, disk and download estimates shown up front. GLM-5.3-Flash and Nemotron 3.5 Super VL 120B GGUF rentals boot from a prebuilt CUDA 13 llama.cpp engine image (ghcr.io/whq02/mercury-llamacpp) so pod setup skips source compilation. Runpod rentals, background setup workers and 5-second stock watches are joined by Vast.ai and Lambda rental adapters, which are new in 1.0.0-beta.40 and verified against the vendors' published APIs on fixtures (live rental runs were not performed). For machines you own, /llmhost → My computers and mercury host inspect this computer or a saved SSH machine, pick a memory-fitting Ollama or GGUF recipe, verify a local SHA256SUMS weight bank or confirm a download, and run the server as a user login service registered as compat/box-<name>/<model>.
Another practical pain point was installing a new Mercury build while a long chat and background daemon were already running on the older build. Live chats now show a one-time notice between turns when a newer build is installed, and /restart moves the live session onto the installed build without stranding the conversation: it lists the Monitor watches and background shells that will stop, stops the session's runner before handover, switches the background daemon to the installed build, and resumes the chat where you left off.
On the model and runtime side, /effort now lists and sends supported reasoning levels for self-hosted and OpenAI-compatible servers (chat_template_kwargs on llama.cpp and top-level reasoning_effort on vLLM, SGLang, Ollama and Kimi K3, with a one-retry fallback on HTTP 400). llama.cpp conversations reuse a consistent server slot (id_slot with cache_prompt: true), self-hosted servers poll liveness during long prompt prefills instead of timing out after 30 seconds, and streaming tool-call arguments count as live crewmate activity. Interactive terminal sessions now run as oculus so permission asks and questions wait for you at the screen without timing out, while unattended mercury run, mercury runner, scheduled and coordinator sessions run as headless and end blocked with the question quoted on the outcome row if an AskUserQuestion goes unanswered.
Finally, running ! shell commands now stream a live output tail in chat with a ticking timer, expanding a connected server in /mcp opens a filterable tool list and per-tool parameter schema view (and pressing ↵ on a needs sign-in row authenticates from the active runner), daemon-hosted chats keep post-compaction messages in order across reattach and /restart, compaction reserves the largest token underestimate seen in the session so file-heavy turns do not immediately re-trigger compaction, and Windows Monitor, run_in_background and inherit_session_env calls fall back to per-command brush processes on machines without Git Bash. The full list of changes and fixes is in the 1.0.0-beta.40 release notes.