Models

https://kipp.ly/transformer-taxonomy/ https://kipp.ly/transformer-inference-arithmetic/

Transformer

• Transformer: A Novel Neural Network Architecture for Language Understanding

• The Illustrated Transformer

• Transformer模型详解（图解最完整版）

• 详解Transformer （Attention Is All You Need）

• The Annotated Transformer

• Introduction to Attention Mechanism

• The Transformer Attention Mechanism

• Transformers Explained Visually - Overview of Functionality

• Transformers Explained Visually - How it works, step-by-step

• Transformers Explained Visually - Multi-head Attention, deep dive

• 分析transformer模型的参数量、计算量、中间激活、KV cache

• 为什么现在的LLM都是Decoder-only的架构？

Position Encoding

• Transformer中的position encoding
• 如何理解Transformer论文中的positional encoding，和三角函数有什么关系？

multi-head

• BERT 可解释性-从"头"说起
• 为什么Transformer 需要进行 Multi-head Attention？

swin-transformer

• https://github.com/microsoft/Swin-Transformer

transformer performance optimization

• https://github.com/NVIDIA/FasterTransformer

• NVIDIA BERT推理解决方案Faster Transformer开源啦

• 英伟达Fastertransformer源码解读

• https://github.com/Dao-AILab/flash-attention

KV cache

• 大模型推理加速：看图学KV Cache

Flash Decoding

• 大模型推理加速之Flash Decoding：更小子任务提升并行度

llama

• Llama 2详解

Frameworks

llama.cpp

• https://github.com/ggerganov/llama.cpp

• llama.cpp源码解析--CUDA流程版本

• 笔记：Llama.cpp 代码浅析（一）：并行机制与KVCache

• 笔记：Llama.cpp 代码浅析（二）

• 笔记：Llama.cpp 代码浅析（三）：计算开销

llama2.c

• https://github.com/karpathy/llama2.c

Optimization

• Mastering LLM Techniques: Inference Optimization