techtree / Linear algebra

Dot product

Multiply matching entries and add them up; one number that measures agreement.

40 seconds

\begin{bmatrix} 3 \\ 1 \end{bmatrix} \cdot \begin{bmatrix} 2 \\ 4 \end{bmatrix} = 3 \cdot 2 + 1 \cdot 4 = 10

Transcript

Here are two vectors. Three, one. And two, four.

The dot product multiplies matching entries. Three times two is six. One times four is four.

Then it adds them up. Six plus four is ten.

Two vectors go in. One number comes out.

Draw them as arrows, and that number says how much they agree.

Pointing the same way, it is big and positive.

Turn one away, and it shrinks. At a right angle, it is zero.

Pointing in opposite directions, it goes negative.

Multiply, add, and out comes one number: how much two vectors agree.

Needs

Length, angle, and cosine similarity video
Length comes from the dot product with itself; dividing out lengths leaves the angle.
Matrix times vector video
Each output entry is a dot product of one row with the vector.
Attention scores video
Dot each query with every key, then softmax into weights.