Home - ufal/NPFL095 GitHub Wiki
Modern Methods in Computational Linguistics NPFL095
See Requirements for getting credits. Let me know if you are interested in this course.
Autumn&Winter 2025/2026
Monday, 17:20-18:50, S6
| date | speaker | paper |
|---|---|---|
| Sep 29 | startup meeting | |
| Oct 6 | Martin Popel | Sravana Reddy, Kevin Knight: What We Know About The Voynich Manuscript, ACL 2011. |
| Oct 13 | Martin Popel | Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean: Efficient Estimation of Word Representations in Vector Space, ICLR 2013. Questions |
| Oct 20 | Jan Pavelka | Ashish Vaswani et al.: Attention Is All You Need (The Transformer paper) (blog), NIPS 2017. Questions Video |
| Oct 27 | -- | Turing Game (online) |
| Nov 3 | Hanseo Yoo | Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2018. Questions |
| Nov 10 | Diana Pavlenok | Schick et al.: Toolformer: Language Models Can Teach Themselves to Use Tools, 2023. Questions |
| Nov 17 | no seminar | Struggle for Freedom and Democracy Day |
| Nov 24 | Roman Tomchik | Sebastien Bubeck et al.: Sparks of Artificial General Intelligence: Early experiments with GPT-4, 2023. Chapters: 8, 9.3(Bias),10 Questions |
| Dec 1 | Alina Haitota | Elena Voita, Rico Sennrich, Ivan Titov: The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives, 2019. Questions. See also a blog. |
| Dec 8 | Rob Chiocchio | Sennrich, Haddow, Birch: Neural Machine Translation of Rare Words with Subword Units, 2015 Questions |
| Dec 15 | Benjamin Reeves | Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu: BLEU: a Method for Automatic Evaluation of Machine Translation, ACL 2002. Questions |
| Jan 5 | Kornelia Skorupinska | Thibault Sellam, Dipanjan Das, Ankur Parikh: BLEURT: Learning Robust Metrics for Text Generation, 2020 |
Papers suggestions
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu: BLEU: a Method for Automatic Evaluation of Machine Translation, ACL 2002. Questions
- Schick et al.: Toolformer: Language Models Can Teach Themselves to Use Tools, 2023. Questions
- Sennrich, Haddow, Birch: Neural Machine Translation of Rare Words with Subword Units, 2015 Questions
- Sebastien Bubeck et al.: Sparks of Artificial General Intelligence: Early experiments with GPT-4, 2023. Chapters: 8, 9.3(Bias),10 Questions
- Lester et al: The Power of Scale for Parameter-Efficient Prompt Tuning, 2022. A very interesting paper with a nice 10 minutes video summary.
- Krishna et al.: Downstream Datasets Make Surprisingly Good Pretraining Corpora, 2022
- Weiss, Goldberg, Yahav: Thinking Like Transformers, 2021
- Ivan Provilkov, Dmitrii Emelianenko, Elena Voita: BPE-Dropout: Simple and Effective Subword Regularization, 2020 Questions
- Elena Voita, Rico Sennrich, Ivan Titov: The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives, 2019. Questions. See also a blog.
Papers suggested by students
Add your suggestions here
NN
- TranformerXL (blog)
- Why Self-attention?
- Insertion Transformer
- Music Transformer
- Sequence-Level Knowledge Distillation
- Goodfellow I. et al.: Generative Adversarial Networks
- Neural Machine Translation with Byte-Level Subwords
Dependency parsing and Universal Dependencies
- Chen and Manning: A Fast and Accurate Dependency Parser using Neural Networks
- Ammar, W. et al.: Many Languages, One Parser
- Dozat and Manning: Deep biaffine attention for neural dependency parsing (winner of CoNLL-2017)
LLM
- Zhang et al.: Multimodal Chain-of-Thought Reasoning in Language Models
See also papers covered in Past semesters, especially those on structured prediction (M. Collins et al., A.McCallum et al., J. Lafferty et al.). Unless otherwise stated, teaching materials for this course (all content on this wiki) are available under CC BY-SA 4.0.