Embeddings
Close = similar.
Math on ideas.↘
What is an embedding, and why is a raw token ID not enough?
An embedding replaces a token's ID with a learned dense vector, a long list of numbers, say 384 or 4096 of them. A raw ID like 4127 is just a name-tag: it carries no meaning and 893 isn't "less than" 4127 in any useful sense.
The embedding packs meaning into the numbers themselves, arranged so similar meanings sit close together. That's what lets the model do math on ideas instead of matching exact spellings.
than token 4127?”
→ No. Raw IDs carry no meaning.
Only the learned vectors do.
What does 'similar meanings sit close together' actually mean,
and how do you measure it?
It means the vectors are placed so that words used in similar ways end up as nearby points: "cat", "dog" and "puppy" cluster together, far from "car". You measure how close with cosine similarity, which roughly asks whether the two vectors point the same direction.
A score near 1 means very similar, and a score near 0 means unrelated. This is why a model can treat synonyms almost interchangeably without ever being told they're synonyms.
Explain the king − man + woman ≈ queen example.
Directions in the space are meaningful, not just distances. The arrow from "man" to "woman" turns out to be about the same as the arrow from "king" to "queen", a consistent gender direction.
So if you take the vector for king, subtract man and add woman, you walk along that direction and land almost exactly on queen.
It shows relationships are encoded as parallel directions, which is why models generalize instead of memorizing each word in isolation.
This is a small sample.
600+ interview questions live inside Skillumen.
Every concept card carries its own interview questions, graded as you answer.
Free to start.