Similarity Search

Once your text is turned into embeddings, finding related content means finding vectors that are close together. You need a way to measure "close". The standard measure is cosine similarity.

Cosine similarity#

Cosine similarity compares the direction of two vectors, not their length. It is the cosine of the angle between them.

The closer the score is to 1, the more similar the meaning.

Why direction and not distance#

You could measure the straight-line distance between two points instead. The problem is that a vector's length can change for reasons that have nothing to do with meaning, such as how long the document is. Two texts about the same thing could sit far apart only because one vector is longer.

Cosine similarity divides out the length, so only the direction counts. That direction is what carries the meaning.

Cosine and dot product#

When every vector is normalized to length 1, cosine similarity and dot product give the same ranking. Many systems normalize once, when they store the vectors, and then use the dot product because it is faster to compute.

import math

def normalize(v: list[float]) -> list[float]:
    length = math.sqrt(sum(x * x for x in v))
    return [x / length for x in v]

def dot(a: list[float], b: list[float]) -> float:
    return sum(x * y for x, y in zip(a, b))

def cosine(a: list[float], b: list[float]) -> float:
    # Dividing by both lengths removes length from the score.
    return dot(a, b) / (math.sqrt(dot(a, a)) * math.sqrt(dot(b, b)))

a, b = [3.0, 4.0], [6.0, 8.0]  # same direction, different lengths
print(cosine(a, b))                       # 1.0
print(dot(normalize(a), normalize(b)))    # 1.0, same result

What happens at query time#

Search runs the same steps for every question:

1. Embed the question with the same model used for the documents.
2. Compare the question vector against the stored vectors.
3. Return the top k most similar ones.

Those top k results are your candidate chunks. Comparing against every stored vector is exact, but it gets slow as the collection grows. Vector databases solve that.

Short line#

Similarity search embeds the question and returns the k stored vectors closest to it. Closeness is usually cosine similarity, which compares direction and ignores length; with normalized vectors, the dot product gives the same ranking faster.

Key takeaways#

• Cosine similarity scores 1 for the same direction, 0 for right angles and -1 for opposite directions.
• It compares direction because vector length can vary for reasons unrelated to meaning, like document length.
• With vectors normalized to length 1, cosine similarity and dot product rank results the same way.
• At query time: embed the question, compare with stored vectors, return the top k as candidate chunks.