Which models actually run on MLX

First question: is there a safetensors file?

MLX loads safetensors. Not GGUF, not PyTorch .bin, not ONNX, not GGML.

This is the single most common disappointment, and the reason is that most "run it locally" material is not about MLX at all: it is about llama.cpp, whose GGUF container is not interchangeable with safetensors. GGUF is llama.cpp's container; it carries its own quantisation schemes and its own metadata, and reading it means implementing that container. A model published only as GGUF does not run on MLX, and no amount of configuration changes that.

The practical shortcut: look for a mlx-community version of the model. That organisation republishes models converted to MLX safetensors, usually in several quantisations, and if one exists it is almost always the right thing to download.

A repository with only pytorch_model.bin is also out, which surprises people because the weights are "there". A real example: BAAI/bge-m3 publishes exclusively PyTorch weights, so it does not load — and the reason is the format, not the architecture, which is supported.

Second question: what does model_type say?

Open config.json in the repository — Hugging Face shows it in the browser — and find model_type. That string decides which implementation reads the weights, and if nothing implements it, the model does not load.

Two things about that string are worth knowing, because both cause wrong conclusions.

Aliases exist. mistral is another name for llama as far as the implementation is concerned: the architecture is the same shape, so a Mistral model runs anywhere llama runs, even though nothing about the name suggests it. MLX's own library keeps a remapping table for this, which is the right thing to consult rather than a list somebody typed out.

Vision models hide behind text-model names. gemma3 and qwen2_5_vl have a model_type that looks like a text model's, and checking the architecture first will wave a vision model through. The reliable signals are elsewhere in the repository: a preprocessor_config.json, a vision_config block inside config.json, or a name containing _vl, vision, llava or paligemma. Check for those before you look at the family.

Third question: how is it quantised?

Quantised MLX models carry a quantization block in config.json with the number of bits and the group size. Four-bit is the common case and works well; eight-bit costs more memory and is closer to the original.

The trap here is per-tensor quantisation. Some publishers quantise different tensors at different precisions and record that as a map inside the config rather than a single setting. A runtime that applies one quantisation to the whole model will read those weights with the wrong number of bits — and this does not fail loudly. It produces numbers that are wrong in a way that still looks like text.

A real example that trips this: Qwen3.8-27B-Uncensored-OptiQ-4bit. The name says 4-bit; the config says otherwise for some tensors.

Fourth question: does the config declare anything exotic?

Three fields cause silent wrongness rather than clean failure, which is why a careful runtime refuses them instead of guessing.

rope_scaling describes how positional information is stretched to reach a longer context. default, llama3 and yarn are the schemes commonly implemented. Something else means the positions will be computed differently from how the model was trained, and the output will be plausible and degraded rather than obviously broken.

Worth knowing about YaRN specifically: when it declares mscale_all_dim, it scales the attention as well as the frequencies, not just the frequencies. Missing that does not produce an error — it produces logits that are all off by the same factor, so the ranking still looks sensible while the values are not.

sliding_window limits how far back attention reaches. Here is the trap, and it is a good one: Qwen2.5 declares sliding_window: 131072 and then sets use_sliding_window: false. A check that reads only the first field rejects a model that is completely fine. You have to read both.

layer_types appears on models that mix attention kinds across layers. Anything other than full attention on every layer means the model expects behaviour that a simple implementation does not provide.

Fifth question: will it fit?

MLX runs on the GPU, and on Apple Silicon the GPU shares memory with everything else. The ceiling is not the RAM figure on the box: it is the working set Metal recommends, which is lower. On a 24 GB machine that came back as 23347 MB — and that is the ceiling for everything, not just for weights.

A rough estimate that holds up: the weights on disk are close to the memory they occupy, and a long prompt costs meaningfully more on top. Measured on a small llama, peak memory with a 4096-token prompt was 1.68× the weights. Doubling the weight size is a safe planning figure.

So a 4-bit model whose files total 8 GB wants around 16 GB available to be comfortable with long prompts — which on a 16 GB machine means it is the only thing you are running.

Reading a model card without being misled

Model cards are written by people who are pleased with their model, which is fair, and they are not written to tell you whether it will load. Three habits help.

Ignore the parameter count in the name and look at the file sizes. "7B" tells you about the architecture, not about what you will download. The same 7B model exists as 15 GB of 16-bit weights, 8 GB at 8-bit and 4 GB at 4-bit, and the name is identical in all three. The file list is the honest number.

Ignore benchmark tables entirely for this purpose. They say nothing about whether the format is right, and a model that scores well and does not load scores nothing on your machine.

Look at what the repository actually contains rather than what the card describes. A card frequently describes the original model while the repository holds a conversion, or the other way around. The files do not have opinions.

When the answer is "not yet"

There is a middle answer between "runs" and "does not run", and it is worth understanding because it is where a lot of models land.

MLX's own library implements well over a hundred architectures. Any given runtime implements fewer — Runlix implements six — and the difference is not a defect on either side: a translated architecture has to be verified against a reference before it can be trusted, and verification is what takes the time.

So there are two lists worth checking rather than one: the runtime's own architectures, and the library's. A family on the second and not on the first is not yet — known, translatable, and waiting to be verified. A family on neither is a different conversation: nobody runs it on MLX today, and no amount of waiting on one runtime changes that.

What none of this tells you

All of the above is about whether the weights will load and produce numbers. It says nothing about whether the model is any good, whether it speaks your language well, whether its licence allows what you intend, or whether the quantisation has cost it something that matters for your task.

Four-bit quantisation in particular is not free. It is usually a fine trade, and on small models it is sometimes not — the smaller the model, the more each bit was doing. If a 4-bit model disappoints you, trying the 8-bit version of the same thing is a cheaper experiment than changing model.

The short version

Before you download, check five things in this order:

1. safetensors present — otherwise stop;
2. not a vision model — check preprocessor_config.json and vision_config first, because the name will not tell you;
3. model_type is implemented — remembering that aliases exist and mistral means llama;
4. quantisation is one setting, not a map per tensor;
5. rope_scaling, sliding_window and layer_types are ordinary — and read both sliding window fields.

Why "I do not know" is a real answer

Every one of these checks can fail to reach a conclusion: the repository might not answer, or might have no config.json at all. The temptation is to guess, because a verdict feels more useful than a shrug.

It is not. A runtime that guesses when it cannot tell will eventually guess wrong, confidently, and you will spend the debugging session on the wrong hypothesis. Not knowing is worth saying out loud.

Runlix runs these checks for you, on the row, before the download — with the reason written next to the verdict, including when the reason is that it could not find out. But the checks are the point, not the tool: they are five questions about a file that anyone can read, and knowing them is worth more than any application that answers them on your behalf.