멘토님과 Q&A - KR-HappyFace/meetup-logs GitHub Wiki
- Q1. Electra에서 pretrain을 할 때, Generator도 train시키는 건가 아니면 discriminator만 train시키는 건가요?
- Q2. KoNLPy에서 mecab이 일본어 tokenizing 기반으로 만들어진 걸로 아는데, 이게 제일 빠르고 성능이 좋은 이유는 뭘까?
NLP Papers listing
Follows Chronological Order
- Attention is All you Need | 2017
- GPT | 2018
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding | 2019
- GPT 2: Language Models are Unsupervised Multitask Learners | 2019
- XLNet: Generalized Autoregressive Pretraining for Language Understanding | 2019
- RoBERTa | 2019
- ERNIE: Enhanced Representation through Knowledge Integration | 2019
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations | 2020
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators | 2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension | 2020