How the model represents each token inside the network, before attention mixes them.
Made for you now: slide lessons from basics to production, usually with code and a quiz. See a finished one
An embedding model turns text (or an image) into a list of numbers, a vector, typically a few hundred to a few thousand long. Texts with similar meaning end up close together in that space.
The model learns this geometry by being trained so that related items sit near each other and unrelated ones sit far apart.
Cosine similarity compares the angle between two vectors and ignores their length: 1 means same direction, 0 unrelated, -1 opposite. Dot product is the same idea when vectors are normalised.
Use the metric the embedding model was trained for. Mixing metrics quietly degrades results.
Embeddings capture topic well and exact detail poorly: product codes, names and numbers can be missed. Queries and documents must be embedded with the same model, and changing the model means re-embedding everything.
Step 1
Text
A sentence, paragraph or query.
Step 2
Embedding model
Maps the text to a fixed-length vector.
Step 3
Vector
A point in high-dimensional space.
Step 4
Similarity
Cosine or dot product against stored vectors.
Step 5
Nearest items
The closest vectors are the most related text.