Skip to content

LLM chat gateway

POST /api/llm/chat proxies a configured LLM provider and streams the response as llm_chunk (then llm_done/llm_error) events over the caller’s existing GET /api/realtime connection — no separate streaming transport. An optional collection config persists the exchange to a collection of your choosing. Because every call spends the operator’s own provider quota, add an explicit rate limit on this exact path in production; authentication alone doesn’t bound cost the way it does for an ordinary record write.

See Reference → REST API → LLM for the full request/response shape, including why the HTTP response itself is always the final result (never a chunked/streamed body) and how an optional collection persists the exchange subject to that collection’s own createRule.