Run the gateway

Build and start warpllm-server with Docker or cargo, then make a request.

The HTTP gateway has landed on main but is not part of a published release. To run it today you build it from source or from the Dockerfile in the repository root. Flags, endpoints, and the error envelope may still change before it ships.

Docker

The repository ships a Dockerfile that builds a distroless image running as a non-root user.

1. Build the image

   git clone https://github.com/warpllm/warpllm.git
   cd warpllm
   docker build -t warpllm-server .
   
2. Run it with your provider keys

   docker run --rm -p 8080:8080 \
     -e OPENAI_API_KEY \
     -e DEEPSEEK_API_KEY \
     warpllm-server
   

   Defaults bind 0.0.0.0:8080. Append flags to override:

   docker run --rm -p 9090:9090 -e OPENAI_API_KEY \
     warpllm-server --port 9090
   
3. Check it is up

   curl http://localhost:8080/health
   

   { "status": "ok", "version": "0.6.1" }
   

The image has no shell and therefore no HEALTHCHECK — distroless is deliberate. Point your orchestrator’s probe at GET /health instead. TLS roots are bundled with rustls, so there is no ca-certificates layer either.

Cargo

git clone https://github.com/warpllm/warpllm.git
cd warpllm

OPENAI_API_KEY=sk-... cargo run --release -p warpllm-server

Requires Rust 1.85 or newer.

Docker Compose

services:
  warpllm:
    build: .
    ports:
      - "8080:8080"
    environment:
      OPENAI_API_KEY: ${OPENAI_API_KEY}
      DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}
      MISTRAL_API_KEY: ${MISTRAL_API_KEY}
      MOONSHOT_API_KEY: ${MOONSHOT_API_KEY}
      OPENCODE_API_KEY: ${OPENCODE_API_KEY}
      OPENROUTER_API_KEY: ${OPENROUTER_API_KEY}
    command: ["--stream-read-timeout-secs", "90"]

Keys are read once, at startup. Changing a variable requires a restart.

Your first request

curl

curl http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5-nano",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

No Authorization header is needed — the gateway ignores it and uses its own provider keys. See Connecting a client for the OpenAI SDKs.

Deployment notes

Binding and exposure

The default --host 0.0.0.0 listens on every interface. Since the gateway has no caller authentication, bind it to a private interface (--host 127.0.0.1) or keep it inside a private network and put your own auth in front.

Health checks

GET /health returns 200 with {"status": "ok", "version": "..."}. It reports that the process is serving, not that any provider is reachable — warpllm sends no upstream traffic to answer it.

Timeouts under a proxy

Streams are long-lived server-sent events. A reverse proxy with a short read timeout will cut them off before warpllm does. Set the proxy’s timeout above --timeout-secs, and disable response buffering for /v1/chat/completions.

Scaling

The gateway holds no per-caller state — every request carries its own conversation — so instances scale horizontally with no coordination.

Next

Configuration

Every flag and what it defaults to.

Connect a client

Point the official OpenAI SDKs at it.