AI scientist - chunhualiao/public-docs GitHub Wiki

information retrieval

workshops and conferences

The first academic workshop explicitly aimed at accepting only AI-generated papers is Agents4Science 2025, organized by researchers at Stanford University. This conference solicits papers where AI is the primary author, with humans acting as advisors, and initial reviews are conducted by AI systems. It was announced to assess the potential and limitations of AI-generated scientific work, as current venues often do not allow such submissions.

Challenges in each step

  • idea generation:
  • conduct experiments
  • write the paper
    • related work section
    • manage citations: find related work papers, create/copy bibtext entries, then cite it in the text, BibTex Manager Extension of VS Code

Existing Autonomous AI Scientist Systems

https://www.futurehouse.org/research-announcements/launching-futurehouse-platform-ai-agents

The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024a. The AI Scientist framework generates novel research ideas, writes code, conducts experiments, and creates a full scientific paper with an automated peer-review system to evaluate the work.

AI-Researcher: Fully-Automated Scientific Discovery with LLM Agents

  • github.com/HKUDS/AI-Researcher

Agent Laboratory: Using LLM Agents as Research Assistants

Researchagent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738, 2024.

  • automatically generates research ideas, methods, and experiment designs, iteratively refining them through feedback from multiple reviewing agents that mirror peer discussions and leverage human-aligned evaluation criteria to improve the outputs.

SCIAgents: automating scientific discovery through multi-agent intelligent graph reasoning, Sep, 2024

  • Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers. arXiv preprint arXiv:2409.04109, 2024.

"AI Tools for Research: Recommendations from Nature"

Five main categories, each with specific tools mentioned in the Nature article:

  1. Literature Review Tools:

    • Gemini Deep Research - Conducts in-depth searches over 30 minutes with text, figures, and proper citations
    • OpenAI Deep Research - Advanced active learning for comprehensive literature exploration
    • SciSpace - "Chat with PDF" function for deep dives into specific papers
    • Other tools mentioned: Claude, NotebookLM, PDF.ai
    • check academic literature using either Semantic Scholar or OpenAlex APIs

  2. Hypothesis Generation:

    • Research Rabbit - Visualization tool that generates interconnected webs of research by topic, author, or methodology
    • Allen Institute Tools - Upcoming hypothesis generation products combining ideas across papers into novel concepts
    • Elicit - Research assistant for generating experiments based on hypotheses

  3. Experimental Assistants:

    • CRESt - Copilot for Real-world Experimental Scientist - helps craft and run experiments with robotic integration
    • Scite - AI research assistant for experiment planning and literature analysis
    • ChatGPT for Troubleshooting - Helps brainstorm explanations and debug experimental issues

  4. Code & Statistics Tools:

    • GitHub Copilot - AI pair programmer for coding data analysis and visualizations
    • Amazon CodeWhisperer (Amazon Q Developer) - Code generation with security scanning capabilities
    • Cursor - Editor that makes data analysis easier with visualization capabilities
    • CatalyzeX - Locates and shares existing code from published papers

  5. Scientific Writing Aids:

    • Paperpal - Checks academic manuscripts against journal submission guidelines
    • Thesify - Multi-stop platform for students with focus on grant writing and paper formatting
    • SciSpace - Templates for research proposals, literature reviews, and abstracts
    • Coral AI/Quillbot/Whisper - Translation capabilities covering dozens of languages

AI tools are spotting errors in research papers: inside a growing movement, Nature, 2025