[layer norm] layer normalization - dsindex/blog GitHub Wiki

layer normalization

• Layer Normalization(original paper)
• Weight Normalization and Layer Normalization Explained

implementation

• layer_norm() in tensorflow
  • normalize [begin_norm_axis ~ R-1] dimensions
• transformer/modules.py
  • normalize [R-1] dimension only