MoE

1.4 Mathematics of Large Language Models: Training, Inference, Attention, Scaling, and Alignment

We build the mathematics behind modern large language models from scratch: text as probability, tokenization, embeddings, transformers, attention, loss functions, backpropagation, …