Large Language Models - HanjieChen/Reading-List GitHub Wiki
Misc
- Social Skill Training with Large Language Models
- Teaching Large Language Models to Self-Debug
- Prompting with Pseudo-Code Instructions
- Hypothesis Generation with Large Language Models
- Mechanism Design for Large Language Models
- (INTHE)WILDCHAT: 570K CHATGPT INTERACTION LOGS IN THE WILD
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
- MULTITASK PROMPTED TRAINING ENABLES ZERO-SHOT TASK GENERALIZATION
- Scaling Instruction-Finetuned Language Models
- Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
- Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing
- SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
- Satisfiability-Aided Language Models Using Declarative Prompting
- The Power of Scale for Parameter-Efficient Prompt Tuning
- SELF-CONSISTENCY IMPROVES CHAIN OF THOUGHT REASONING IN LANGUAGE MODELS
- Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
- REPLUG: Retrieval-Augmented Black-Box Language Models
- RECITATION-AUGMENTED LANGUAGE MODELS
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
RLHF
- The PRISM Alignment Project: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
- Pretraining Language Models with Human Preferences
- Training language models to follow instructions with human feedback
- Pinpoint, Not Criticize: Refining Large Language Models via Fine-Grained Actionable Feedback
Efficiency
- ReFT: Representation Finetuning for Language Models
- LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models
- Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
Agents
- Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- Generative Agents: Interactive Simulacra of Human Behavior
- AgentGPT
- BabyAGI
- Auto-GPT: An Autonomous GPT-4 Experiment
- AGENTBENCH: EVALUATING LLMS AS AGENTS
Challenges and Open Questions
- Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?
- Modern language models refute Chomsky’s approach to language
- Large Language Models are not Fair Evaluators
- Do Language Models Know When They're Hallucinating References?
- Scalable Extraction of Training Data from (Production) Language Models
- WHEN DOES IN-CONTEXT LEARNING FALL SHORT AND WHY? A STUDY ON Specification-Heavy TASKS
- Deceiving Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?
- Are Emergent Abilities of Large Language Models a Mirage?
- Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder
- When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities
- CERTIFYING LLM SAFETY AGAINST ADVERSARIAL PROMPTING
- When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
- Are Emergent Abilities of Large Language Models a Mirage?
Evaluation
- ThoughtSource: A central hub for large language model reasoning data
- Faithful Chain-of-Thought Reasoning
- Data Portraits: Recording Foundation Model Training Data
- tinyBenchmarks: evaluating LLMs with fewer examples
- NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist
- Holistic Evaluation of Language Models
- EVALUATING LARGE LANGUAGE MODELS AT EVALUATING INSTRUCTION FOLLOWING
- A survey on evaluation of large language models
Long-horizon
- MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation
- Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
- MemoryBank: Enhancing Large Language Models with Long-Term Memory
- Equal Long-term Benefit Rate: Adapting Static Fairness Notions to Sequential Decision Making
Trustworthiness
- Enabling Large Language Models to Generate Text with Citations
- Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4
- Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
- Factuality Enhanced Language Models for Open-Ended Text Generation
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents
- REPRESENTATION ENGINEERING: A TOP-DOWN APPROACH TO AI TRANSPARENCY
- Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
- What Does it Mean for a Language Model to Preserve Privacy?
- Perils of Self-Feedback: Self-Bias Amplifies in Large Language Models
- Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate
- Benchmarking LLMs via Uncertainty Quantification
- Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning
- Towards Automated Circuit Discovery for Mechanistic Interpretability
- TRUSTLLM: TRUSTWORTHINESS IN LARGE LANGUAGE MODELS
- Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations
- Explaining Machine Learning Models with Interactive Natural Language Conversations Using TalkToModel
- Rethinking Explainability as a Dialogue: A Practitioner’s Perspective
- INTERFAIR: Debiasing with Natural Language Feedback for Fair Interpretable Predictions
Editing
- Locating and Editing Factual Associations in GPT
- Unified concept editing in diffusion models
- Black-Box Access is Insufficient for Rigorous AI Audits
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
- Locating and Editing Factual Associations in Mamba
Safety/Privacy
Psychology and Personalization
- WHO IS CHATGPT? BENCHMARKING LLMS’ PSYCHOLOGICAL PORTRAYAL USING PSYCHOBENCH
- From Persona to Personalization: A Survey on Role-Playing Language Agents
- Personality Traits in Large Language Models
- Dialect prejudice predicts AI decisions about people’s character, employability, and criminality
- Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models
- How well can an AI chatbot infer personality? Examining psychometric properties of machine-inferred personality scores
- Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5
- Using Artificial Populations to Study Psychological Phenomena in Neural Models