Skip to content

wavecat-sdk

wavecat-sdk

Run wavecat's heavy model on your own hardware. wavecat-sdk is a tiny, always-on gateway that presents a stable OpenAI-compatible endpoint on localhost and forwards every request to your model server (llama.cpp, vLLM, …). Point wavecat at the gateway and it routes the interactive heavy work — chat, agent, and code turns — there instead of the bundled local 35B.

wavecat ──/v1──► [wavecat-sdk gateway :8800] ──/v1──► your llama.cpp / vLLM

pip install wavecat-sdk
wavecat-sdk serve --upstream http://127.0.0.1:8000/v1 --model my-model

Why pip install wavecat-sdk?

  • Your hardware, your model. Offload wavecat's heavy turns to a GPU box, workstation, or any OpenAI-compatible server you already run — no rewrites on either side.
  • One command, always on. It's a CLI: install it, run wavecat-sdk serve, and leave it running. Nothing to wire up in code.
  • Minimal exposure by design. The gateway only ever sees OpenAI chat-completion payloads — prompts, tool schemas, tool results as text, and sampling params. It never runs a wavecat tool and never touches your local data. Tools always execute inside wavecat; this process only generates tokens.
  • Drop-in OpenAI compatibility. Speaks /v1/chat/completions, /v1/models, and /health. Streaming SSE frames are relayed through unchanged, so wavecat sees the exact OpenAI deltas it expects.
  • Graceful fallback. If the gateway or your upstream is ever unreachable, wavecat silently falls back to the local 35B, so turns never hard-fail.
  • Typed and tiny. ~250 lines, three dependencies (fastapi, uvicorn, httpx), ships a py.typed marker, and is covered by tests.

What it is not

It is not a client library you import to call an LLM, and it is not an inference server. It's the thin, sanitizing bridge that sits between wavecat and the model server you bring.

Next steps