Model file format

When you will meet it

You hit this the first time you download a model from Hugging Face or the community and try to run it in Ollama, LM Studio or your own script: pick the wrong format and the tool either says "model not found" or throws a wall of shape errors you mistake for bugs in your own code. Knowing which format belongs to which runtime is the first gate of local LLMs.

Minimal example

同一個模型的三種常見包裝：

  model.safetensors      PyTorch 生態、vLLM 等伺服器框架
                         （通常未量化；權重與 metadata 分開存放）
  model-Q4_K_M.gguf      llama.cpp 系：Ollama、LM Studio
                         （單一檔案自帶 tokenizer，檔名就標了量化等級）
  MLX 版本               Apple Silicon 的 MLX 生態

下載前先問一句：我要用的那個工具，吃哪一種？

Notice GGUF filenames usually carry the quantization grade (Q4, Q5, Q8...) — format and quantization arrive together, which is why "wrong file" and "wrong precision" are often the same mistake. One design goal of safetensors is safe loading: unlike older pickle-based formats, loading one cannot execute arbitrary code.

What people get wrong

• Grabbing a link from tool A's download page, feeding it to tool B, and debugging for hours. A format mismatch is a format mismatch — confirm what your runtime supports first, then find the matching file; the reverse order burns bandwidth and sanity.
• Assuming different formats mean different models, or that one format is inherently "better". Usually it is the same weights in different boxes; what differs is ecosystem support, quantization options and loading efficiency. Pick the format your runtime wants, not a leaderboard winner.