attention mechanisms

Sebastian Raschka’s LLM Architecture Gallery

Raschka’s gallery compares GPT-2, Llama 3, OLMo 2, DeepSeek, and Qwen stacks with exact layer, cache, and attention data.