Home - ufal/NPFL095 GitHub Wiki
Modern Methods in Computational Linguistics NPFL095
See Requirements for getting credits. Let me know if you are interested in this course.
Autumn&Winter 2026/2027
Monday, 16:00-17:30, S1
| date | speaker | paper |
|---|---|---|
| Oct 5 | startup meeting, announce Turing Game | |
| Oct 12 | Martin Popel | Sravana Reddy, Kevin Knight: What We Know About The Voynich Manuscript, ACL 2011. |
| Oct 19 | Maksym Khavil | Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean: Efficient Estimation of Word Representations in Vector Space, ICLR 2013. Questions |
| Oct 26 | Martin Popel | Ashish Vaswani et al.: Attention Is All You Need (The Transformer paper) (blog), NIPS 2017. Questions Video |
| Nov 2 | Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2018. Questions | |
| Nov 9 | Karel Podzimek | |
| Nov 16 | Turing Game (no class, homework) | |
| Nov 23 | Tomas Horych | DeepSeek-AI: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning |
| Nov 30 | Aeirya Mohammadi | |
| Dec 7 | Antonín Otmar | |
| Dec 14 | ||
| Jan 4 | Martin Popel | Turing Game results |
Papers suggestions
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu: BLEU: a Method for Automatic Evaluation of Machine Translation, ACL 2002. Questions
- Schick et al.: Toolformer: Language Models Can Teach Themselves to Use Tools, 2023. Questions
- Sennrich, Haddow, Birch: Neural Machine Translation of Rare Words with Subword Units, 2015 Questions
- Sebastien Bubeck et al.: Sparks of Artificial General Intelligence: Early experiments with GPT-4, 2023. Chapters: 8, 9.3(Bias),10 Questions
- Lester et al: The Power of Scale for Parameter-Efficient Prompt Tuning, 2022. A very interesting paper with a nice 10 minutes video summary.
- Krishna et al.: Downstream Datasets Make Surprisingly Good Pretraining Corpora, 2022
- Weiss, Goldberg, Yahav: Thinking Like Transformers, 2021
- Ivan Provilkov, Dmitrii Emelianenko, Elena Voita: BPE-Dropout: Simple and Effective Subword Regularization, 2020 Questions
- Elena Voita, Rico Sennrich, Ivan Titov: The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives, 2019. Questions. See also a blog.
Papers suggested by students
Add your suggestions here
NN
- Why Self-attention?
- Sequence-Level Knowledge Distillation
- Goodfellow I. et al.: Generative Adversarial Networks
- Neural Machine Translation with Byte-Level Subwords
Dependency parsing and Universal Dependencies
- Chen and Manning: A Fast and Accurate Dependency Parser using Neural Networks
- Ammar, W. et al.: Many Languages, One Parser
- Dozat and Manning: Deep biaffine attention for neural dependency parsing (winner of CoNLL-2017)
LLM
- Zhang et al.: Multimodal Chain-of-Thought Reasoning in Language Models
See also papers covered in Past semesters. Unless otherwise stated, teaching materials for this course (all content on this wiki) are available under CC BY-SA 4.0.