Language Modeling - LiamWahahaha/natural-language-processing GitHub Wiki

N-gram Language Models

  • Maximum Likelihood Estimation
  • Smoothing
  • Backoff
  • Katz Backoff
  • Interpolation
  • Good-Turing Estimate

Generative Models

Discriminative Models

Logistic Regression

Neural Networks and Neural Language Models

Notes

References

  1. Speech and Language Processing[Daniel Jurafsky, James H. Martin, 2018](Chapter 3, 7)
  2. Probabilistic Language Models[Smoth N., 2017]
  3. NLP Course Notes on Language Modeling [Collins M., 2013]
  4. Adversarial Evaluation for Models of Natural Language[Smith N., 2013] (Section 2.4)
  5. erplexity versus error rate for language modeling[Daumé III, 2014]