One endpoint, compatible with OpenAI and with Ollama

What it serves

The two protocols disagree in ways that only show up against real clients, and each disagreement below cost a debugging session to find.

The health check is not decoration: clients of the Ollama API ask for it before anything else, and without the answer they expect they stop with "something went wrong". /api/tags includes a digest because Ollama's own command-line client truncates that field to twelve characters without checking it exists and panics when it does not. /api/show reads both model and name, because the CLI sends both and leaves one empty. Streaming defaults differ by protocol and each follows its own convention: NDJSON on by default for Ollama, off by default for OpenAI. Tool calls differ in shape — arguments is an object in one and a string in the other, and only one carries id and type.

So: compatible with the Ollama API for listing, showing and chatting, tested against Ollama's own client. Not for pull, push, create, copy, delete, ps, embed or version — Runlix downloads and manages its models itself. Any OpenAI-compatible client that needs /v1/chat/completions, /v1/models or /v1/embeddings works.

What you can actually point at it

Anything that speaks one of the two protocols, which by now is most things.

Editor extensions that let you set a base URL — Continue, Cline, Zed's assistant, the OpenAI-compatible mode of a dozen others — need /v1/chat/completions and /v1/models, and both are here. Scripts using the official OpenAI client libraries work by pointing the base URL at http://127.0.0.1:11435/v1 and passing any string as the key, since there is no key to check. Anything that already talks to Ollama finds the endpoints it looks for, in the shapes it expects, because they were tested against Ollama's own command-line client rather than against the documentation.

The port is one above Ollama's on purpose. If you run both, neither has to move, and a client configured for one will not silently reach the other and answer with a model you did not choose.

What you get is the model you named, resolved among the ones you have downloaded. Name one you do not have and the error says which name failed and that it is not among the downloaded ones — not a generic four-hundred that leaves you comparing strings by eye. Send no name at all and you get the loaded model, which is the last one used rather than an arbitrary pick.

What it tells you, and how honestly

Streaming, in both dialects

The two protocols stream differently and each keeps its own convention, because a client written for one expects that one and nothing else.

OpenAI clients get server-sent events: data: lines carrying a JSON fragment each, one identity shared by every chunk of the same answer, and a final data: [DONE]. Ollama clients get newline-delimited JSON objects, one per line, with the last one carrying the counts. Both are sent chunked, so the first words appear while the rest is still being generated.

One exception, and it is deliberate: when tools are in play, streaming holds back. Half a JSON function call arriving in pieces is useless to every client that has ever consumed one, so the decision is sent whole or not at all.

Only localhost, and no authentication

The server binds 127.0.0.1 and nothing else. There is no token, no password, no user.

That is a decision rather than an omission. Authentication that matters is authentication someone maintains, and a checkbox marked "require a key" on a loopback socket mostly provides the feeling of one. If you need this reachable from elsewhere, put a proxy in front and let the proxy decide who gets in — it is the piece that already knows how.

Embeddings, from the same engine

/v1/embeddings turns text into vectors with the same Rust engine that runs the chat models, in the same process — no second server, no Python. It reads the encoder families that the mlx-embeddings library covers for BERT and XLM-RoBERTa, translated and checked against it the same way the chat families are checked against mlx-lm.

The model it uses is the one you pick in Settings, or the first embedding model you have downloaded. With none downloaded it answers 503, "no embedding model loaded" — which means download one, not that the endpoint is missing. The request shape is OpenAI's: a string or a list of strings in, one vector per input out. Too many texts in one request is a 413 that names the limit, not a timeout.

What it does not do

Tools are returned, not executed. Send tools with your request and the model's calls come back to you in the right shape for your protocol; running them is your client's job. The Runlix chat is the one place where Runlix itself runs them.

If a model's chat_template has no idea what tools are, the request still succeeds and a warning goes to the log — the OpenAI protocol has no field for saying "I passed this along and nobody understood it."

The stop parameter is accepted and ignored: the engine implements stop sequences, and the API does not currently pass them down. The Ollama embedding endpoint is not served — embeddings are on the OpenAI paths.

One generation at a time; a second concurrent request is refused rather than queued behind an unpredictable wait.

Putting it behind something

If you do want this reachable from another machine, the shape that works is a reverse proxy on the same Mac, listening where you want it and forwarding to 127.0.0.1:11435. The proxy is where authentication belongs — a bearer token, mutual TLS, an identity provider, whatever your situation calls for — because that is a piece with a maintainer and a threat model, and a checkbox in a settings pane is neither.

Two things to keep in mind if you do. Generation is one at a time, so a shared endpoint will refuse requests that arrive during another; if several people are meant to use it, the proxy needs a queue and a decision about how long a caller waits. And there is no per-caller accounting: the usage numbers in each response are true for that response and are not totalled anywhere.

Chat messages are never flattened

Worth stating because it is the single most common way a local API produces worse answers than the model is capable of. Messages are passed as a structured array and the engine applies the model's own chat_template, because the engine is the piece holding the tokenizer.

Measured on one model, same prompt: with the template the answer is "Paris.". Flattened into role: content lines it becomes "Paris\nassistant: …\nuser: What" — the model keeps inventing turns. It does not look like a bug. It looks like a worse model.