YOUR CURIOSITY. YOUR STARTING POINT.

A course explains it.
Frizzl helps you
understand it.

Start with what you know. Work through one focused idea from real course material, then explain it in your own words.

Start now See how it works

Free pilot · one course to try · one idea at a time

INSIDE A TRANSFORMER · TRY IT

How does attention actually work?

A token’s query is compared with other tokens’ keys. The matches decide how much of each value to mix in.

Change the query. Keep the keys and values fixed.

cat57.6%57.6%

sat28.4%28.4%

mat14.0%14.0%

Query A matches “cat” most strongly. All three values still contribute.

Mixed value [0.718, 0.282]

Show me the maths

For this one head, weights = softmax(qKᵀ / √2), then output = weights × V. Softmax exponentiates the scores and divides by their sum.

Invented vectors, in token order
Token	Key (K)	Value (V)
cat	[2, 0]	[1, 0]
sat	[1, 1]	[0.5, 0.5]
mat	[0, 2]	[0, 1]

Scaled scores: [1.414, 0.707, 0.000].

Two key dimensions give the √2 scale. In a model, learned projections produce Q, K and V. Multiple heads, an output projection and feed-forward layers build on this operation.

Does 57.6% on “cat” mean the model will say “cat” next?

Why this matters now: the KV cache

During standard causal decoding, previous tokens’ keys and values can be saved. Each new query reads that cache instead of rebuilding them. Long contexts make this cache expensive; newer architectures compress or selectively read it.

Caching reuses representations. It does not skip attention or give the model permanent memory. A recent example: DeepSeek V4.1’s cache work ↗

Toy head: three earlier tokens and invented 2D vectors. Full causal attention includes the current position and masks future tokens. This is a worked example, not measured model behaviour. Runs in your browser. Read the transformer paper ↗

A CONVERSATION WITH YOUR UNDERSTANDING

A clear next step, not another pile of tabs.

Choose a subject

Pick a starting point and tell us what you want to understand. No need to plan an entire syllabus.

Learn at your level

A short explanation builds on your background, with examples and links to the source material.

Put it into words

Try a question, get feedback and revisit your reasoning. Your lesson and answers stay saved.

Find your next subject.

Open course material

Applied AI Engineering

Build reliable document systems with retrieval, evidence-backed decisions and practical evaluation.

Explore 24 lessons & four projects →

An original Frizzl course · free practice pack

Language models from scratch

Tokenizers, transformers and the ideas behind modern language models.

Based on Stanford CS336 materials

Linear systems & control

Laplace transforms, feedback and stability—one connection at a time.

Open textbook references

Circuits & amplifiers

From voltage and resistance to transistors and amplifier design.

Open electronics references

Try a lesson. Bring a question.

This pilot gives each verified email one course choice, a growing library of lessons and up to five answer checks per lesson. You can return whenever you like. If you choose the wrong course, contact us for a reset.

AI can make mistakes. Check the linked sources. The pilot does not award course completion or prove long-term mastery.

Start now

A course explains it.Frizzl helps youunderstand it.