techtree / Linear algebra
Adding and scaling vectors
Add vectors entry by entry (tip to tail); scale by multiplying every entry.
Transcript
Here are two vectors. Two, one. And one, three.
To add them, add matching entries. Two plus one is three. One plus three is four.
As arrows, start the second one where the first one ends. The sum goes from the start to where you land.
Add them in the other order, and you land in the same place.
To scale a vector, multiply every entry by the same number. Two times two, one is four, two.
Same direction, twice as long. Times one half, it is half as long.
Times minus one, it turns around.
Adding is tip to tail. Scaling is stretching.
Needs
- Vectors video
A vector is a list of numbers, and also an arrow.
Leads to
- Mean, variance, and normalization video
Shift and scale a vector so its entries have mean 0 and spread 1. - Length, angle, and cosine similarity video
Length comes from the dot product with itself; dividing out lengths leaves the angle. - Matrix times vector video
Each output entry is a dot product of one row with the vector. - Position information video
Add a position vector to each token vector so order is not lost. - Residual connections video
Add a block's output back to its input, so each block makes an edit. - Self-attention video
Each token's output is a weighted average of the value vectors.