Similarity search compares the query vector with stored vectors. Doing that one by one is exact, but it does not scale. With a million chunks, you do not want a million comparisons on every query. Vector databases exist to skip most of that work.
Approximate nearest neighbor search#
Vector databases use approximate nearest neighbor (ANN) search. They build an index structure ahead of time that lets them find the closest vectors without checking all of them.
The trade-off is in the name. Now and then, ANN misses the single best match. In exchange, a search that would touch every vector touches only a small part of the collection, which is a massive speed gain. For retrieval this is almost always worth it, because you take several top results anyway.
Two index types#
Two index types cover most real systems. They skip work in different ways.
HNSW#
HNSW (Hierarchical Navigable Small World) builds a layered graph. The top layers have few vectors and long links, and the bottom layer has every vector with short links. A search starts at the top, walks toward the query, then drops down a layer and repeats, skipping most of the space.
HNSW is fast and accurate. It uses a lot of memory, and building the index is slow.
IVF#
IVF (inverted file index) groups the vectors into clusters, each with a center point. A search finds the clusters whose centers are nearest the query and searches only inside those.
IVF is lighter on memory. It needs enough data to form good clusters, and it takes more tuning, such as how many clusters to create and how many to search.
| Index | How it skips work | Strength | Cost |
|---|---|---|---|
| HNSW | Walks a layered graph toward the query | Fast and accurate | Memory-hungry, slow to build |
| IVF | Searches only the nearest clusters | Light on memory | Needs enough data, more tuning |
Choosing a database#
For most applications, pgvector is the practical choice. It is a Postgres extension that supports both HNSW and IVF indexes, so if you already run Postgres, your vectors live next to the rest of your data.
Dedicated vector databases such as Pinecone make sense at very large scale, or when you want managed infrastructure instead of running it yourself.
Short line#
Comparing a query with every vector is too slow at scale, so vector databases use approximate nearest neighbor indexes: HNSW walks a layered graph, IVF searches only the nearest clusters. pgvector supports both and suits most apps.
Key takeaways#
- Exact search compares with every vector and gets too slow as the collection grows.
- ANN indexes find close vectors without checking all of them, at the cost of sometimes missing the best match.
- HNSW is a layered graph: fast and accurate, but memory-hungry and slow to build.
- IVF clusters vectors and searches the nearest clusters: lighter on memory, but needs data and tuning.
- pgvector supports both and is the default choice, especially on Postgres. Dedicated databases fit very large scale or managed needs.