Agentic Research

Full-text search

Also: 全文檢索 · 全文搜索 · 關鍵字搜尋 · BM25 · keyword search

Traditional keyword search that ranks by whether terms appear, how often, and how rare they are; the classic algorithm is BM25. It does not understand meaning — it counts words.

When you will meet it

It complements vector search; it is not obsolete. For an exact error code, order number, function name or rare proper noun, vector search often misses (such tokens are scarce in training data, so embeddings cannot separate them), while full-text search nails it. In hybrid retrieval it owns the "exact" half.

An analogy

Like Ctrl+F across a pile of files: it finds exactly the characters you type, matching only on identical words. It does not "associate" — type "tomato" as a different word and it finds nothing — but that is its virtue: when you need exactness, it is reliably exact.

Minimal example

查詢:「錯誤碼 ERR-4032 怎麼處理」

全文檢索(BM25):
  直接命中所有出現 "ERR-4032" 這串字的文件 → 精準

向量檢索:
  "ERR-4032" 是個罕見 token,embedding 幾乎沒學過,
  可能把它和 "ERR-4031"、"ERR-5000" 當成差不多 → 抓錯

實務:兩者一起用(混合式檢索),再用 rerank 收束

The crux: full-text search matches words, vector search matches meaning. Queries where "the wording must be exactly right" — error codes, model numbers, names — are full-text search's home turf. That is why mature RAG rarely uses vectors alone; it is almost always hybrid.

What people get wrong

  • Assuming "with vector search you no longer need keywords". The opposite: vectors excel at similar meaning but often fail on exact identifiers; the two complement, not replace, each other.
  • Assuming full-text search "understands" your query. It has no grasp of semantics; it only counts term frequency and rarity. So synonyms, typos and reworded queries can slip past it — that is vector search's strength.

Related terms

Next