OpenAI-compatible API reference

hal0-api mounts /v1/* in src/hal0/api/__init__.py: a public_router (no auth dependency, so a client can GET /v1/models before it has credentials) plus an authenticated router for everything else, plus a separate realtime WebSocket router.

Caution

The /v1/* route module’s own comment states auth was removed and the surface is open on the local network — this predates or is independent of the [security].require_auth / hal0 auth machinery documented in Config schema and CLI. Verify whether require_auth actually gates /v1/* routes, /api/* routes, or both before publishing an unqualified “auth is/isn’t enforced” claim.

Endpoint inventory

Path	Method	Notes
/v1/models	GET	Aggregated catalog. Hides non-text-modality models (image/tts/stt) by default — pass ?show_all=true to see them. Supports X-hal0-Model-Filter header or ?owned_by= query param to scope by owner (hal0 extension, not OpenAI-standard).
/v1/models/{model_id}	GET	By-id lookup — bypasses the modality filter above.
/v1/chat/completions	POST	See request body deviations below.
/v1/completions	POST	Pure passthrough — no hal0-specific body handling at the route layer.
/v1/embeddings	POST	Standard {model, input}. Requests that resolve to the NPU “embed” trio slot bypass the generic dispatcher and post directly to that slot’s own /v1/embeddings.
/v1/rerankings	POST	OpenAI-compat shape name. Rewritten to /v1/rerank on the outgoing leg for llama-server upstreams (llama-server only serves native POST /rerank).
/v1/rerank	POST	Alias, for clients that already speak llama-server’s native shape.
/v1/audio/transcriptions	POST	multipart/form-data. model form field is required — 400 request.missing_model if absent. See deviations below.
/v1/audio/speech	POST	JSON body. model is required (400 request.missing_model if missing/blank) — a deliberate deviation from the dispatcher’s usual default-model fallback.
/v1/images/generations	POST	ComfyUI-backed. Not a passthrough — see Images below.
/v1/realtime	WebSocket	Separate module (realtime.py) — not covered in depth by this pass.

Not implemented

Confirmed absent (no route handler — some appear only as internal capability-routing path strings, not reachable over HTTP):

• /v1/images/edits, /v1/images/variations
• /v1/audio/translations
• /v1/moderations
• /v1/assistants, /v1/threads, /v1/files, /v1/fine_tuning/*, /v1/batches

Request body deviations from standard OpenAI shape

/v1/chat/completions

The body is otherwise standard and forwarded largely as-is, with these hal0-specific behaviors:

• "omni": true — opts into hal0’s OmniRouter tool-calling loop. Stripped from the body before forwarding upstream (strict-mode backends reject unknown fields).
• model may be a chat-slot alias — agent, utility, any enabled LLM slot name, or the legacy agent-hermes — rewritten to the slot’s configured model id before dispatch. A <tag>-FLM catalog id (e.g. gemma4-it-e2b-FLM) is further normalized to the native FLM-served tag (e.g. gemma4-it:e2b), since flm serve only advertises the native tag.
• hal0/* virtual model names are resolved against live slot state before dispatch, and a “lane pin” records which specific slot the caller meant when multiple slots share one model id.
• enable_thinking / chat_template_kwargs — a thinking-policy override; if omitted, defaulted server-side from the model registry’s ModelDefaults.enable_thinking.
• Multiple role: system messages are collapsed into one and hoisted to position 0 before dispatch.

/v1/audio/transcriptions

Raw multipart bytes are forwarded byte-for-byte (not re-encoded); the model field is extracted via regex over the raw bytes rather than request.form(), because Starlette can’t re-read a form after the body has already been consumed. If a non-2xx upstream response would leak internal ffmpeg/subprocess error details, hal0 scrubs it and returns a clean envelope instead:

{
  "error": {
    "code": "audio.unsupported_format",
    "message": "unsupported audio format; expected wav/mp3/flac/ogg/m4a/webm",
    "details": { "upstream_status": "..." }
  }
}

forced to HTTP 415.

/v1/audio/speech

{model, input, voice, speed?, response_format?}. If the client omits voice, speed, or response_format, hal0 seeds them from the serving TTS slot’s persisted Settings → Voice defaults (only filling fields the client didn’t send).

/v1/images/generations

The one endpoint that is not a passthrough — it drives ComfyUIProvider.infer() directly (graph submit + history poll), translating the OpenAI-shape request into a ComfyUI workflow.

Request (subset honored): {"model": <curated id>, "prompt": <required>, "n"?, "size"?, "response_format"?: "url"|"b64_json"}, plus a hal0 extension extra_body: {seed?, steps?, cfg?, negative_prompt?}.

• Missing prompt → 422 image.prompt_required.
• model must be one of the curated image models (e.g. sdxl-turbo, sd-1.5-pruned-emaonly) → 404 image.model_not_curated otherwise.

Response:

{
  "created": 1234567890,
  "data": [{ "url": "/api/images/cache/<uuid>.png" }],
  "_hal0": { "meta": { "...": "..." }, "prompt_id": "...", "upstream": "...", "model": "..." }
}

The _hal0 block is a hal0-specific extra field, not part of the OpenAI response shape.

Response deviations

• FLM (NPU) usage extensions — non-streaming FLM responses may carry usage.decoding_speed_tps / usage.kv_token_occupancy_rate_percentage, fields not present in llama.cpp responses and not part of the OpenAI shape.
• /v1/models default filtering — hides non-text-modality models unless ?show_all=true is passed; GET /v1/models/{id} bypasses the filter.

Model → slot resolution

model in a request body is not used as a raw slot address by default — several resolution stages run first (chat-only rewrites happen at the route layer; every request type goes through the dispatcher):

1. Route-layer chat rewrites (chat only) — slot-alias rewrite, FLM tag normalization, virtual hal0/* name resolution + lane pin, then a pre-dispatch SlotManager.load() call so the model loads under its declared backend/device before the request is forwarded.
2. Dispatcher resolution (Dispatcher.dispatch, all request types), in order:
   1. A loaded container-backed slot that already advertises the model id wins outright (health-ordered).
   2. Exact registry lookup (ModelRegistry.route_for) — includes a check that falls through to a healthy sibling slot if the registered binding points at an ERROR-parked slot serving the same model id.
   3. Passthrough to any upstream whose cached /v1/models already lists the id.
   4. Cold-cache prefetch — a bounded, single-flighted live /v1/models fetch against cold remote upstreams, then a re-check of step 3.
   5. Capability/path routing (last resort) — path fragments map to a capability slot (/embeddings → embed, /rerankings//rerank → rerank, /audio/speech → tts, /images/* → img), then model-name substring hints (embed, rerank) and image-model-id prefixes (sdxl, sd-1.5, sd15, flux), then direct SLOT_ALIASES addressing — a caller can request a slot literally by name as a final fallback.

If model is omitted entirely, the path decides the default: /embeddings → embed, /rerank → rerank, /audio/speech → tts, /images/* → img, everything else → agent.

For the full slot-addressing mechanics and how a slot’s state gates whether it’s eligible for dispatch, see Slot lifecycle and Providers, profiles & devices.

See Streaming for how stream: true is handled.