ai scientist - chunhualiao/public-docs GitHub Wiki

information retrieval

• GAIA benchmark
• https://github.com/bin123apple/InfantAgent

AI scientist:deep research

Prompts

• what are the best open source projects that can automatically conduct computer experiments and write a paper regarding parallelizing a graph layout algorithm using OpenMP?

The first academic workshop explicitly aimed at accepting only AI-generated papers is Agents4Science 2025, organized by researchers at Stanford University. This conference solicits papers where AI is the primary author, with humans acting as advisors, and initial reviews are conducted by AI systems. It was announced to assess the potential and limitations of AI-generated scientific work, as current venues often do not allow such submissions.

• https://agents4science.stanford.edu/

• idea generation:
• conduct experiments
• write the paper
  • related work section
  • manage citations: find related work papers, create/copy bibtext entries, then cite it in the text, BibTex Manager Extension of VS Code

https://www.futurehouse.org/research-announcements/launching-futurehouse-platform-ai-agents

The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024a. The AI Scientist framework generates novel research ideas, writes code, conducts experiments, and creates a full scientific paper with an automated peer-review system to evaluate the work.

• The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
  • https://github.com/SakanaAI/AI-Scientist or https://deepwiki.com/SakanaAI/AI-Scientist
• The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
  • https://github.com/SakanaAI/AI-Scientist-v2

SakanaAI:AI Scientist

AI-Researcher: Fully-Automated Scientific Discovery with LLM Agents

• github.com/HKUDS/AI-Researcher HKUDS:AI Researcher

Agent Laboratory: Using LLM Agents as Research Assistants

Researchagent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738, 2024.

• automatically generates research ideas, methods, and experiment designs, iteratively refining them through feedback from multiple reviewing agents that mirror peer discussions and leverage human-aligned evaluation criteria to improve the outputs.

SCIAgents: automating scientific discovery through multi-agent intelligent graph reasoning, Sep, 2024

• Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers. arXiv preprint arXiv:2409.04109, 2024.

Paper QA: alpharxiv.org

Future-House/paper-qa

• Description: High-accuracy RAG tool for querying scientific documents/PDFs, generating grounded answers with in-text citations; focused on literature.
• Key Features: Agentic workflows for iterative refinement; metadata fetching (citations, journals); full-text search; LLM re-ranking/contextual summarization; CLI (pqa); supports multimodal (images/tables); bundled settings (e.g., high_quality, wikicrow for Wikipedia emulation).
• Automation Level: Automates evidence gathering, answer generation, and validation; aids literature reviews/drafting.
• Community & Metrics: Open for contributions. (Stars: Moderate; Forks: Growing; Last commit: Recent.) Why Best: State-of-the-art for scientific RAG; reduces manual PDF handling; integrates with tools like Semantic Scholar.

https://deepresearch-bench.github.io/

GPT-Researcher — a mature multi-agent “deep research” tool that plans, crawls, aggregates sources, and outputs long research reports with citations (PDF/DOCX/Markdown). Apache-2.0; active releases and strong adoption. Useful as a drop-in “research + draft” microservice. https://github.com/assafelovic/gpt-researcher

STORM (Stanford OVAL) — a modular “knowledge curation” system that researches a topic on the web, auto-outlines, then writes a full-length, Wikipedia-style article with citations; actively maintained, MIT-licensed, with a Python package (knowledge-storm) and a demo. Excellent as a scaffold for Related Work / Background. https://github.com/stanford-oval/storm

langchain-ai/open_deep_research

• Description: Configurable deep research agent for generating reports across model providers and search tools; ranks high on Deep Research Bench.
• Key Features: Supports multiple LLMs/search APIs; LangGraph Studio UI; evaluation on 100 PhD-level tasks; human-in-the-loop options; legacy multi-agent/parallel processing.
• Automation Level: Automates research synthesis and report writing; iterative for depth.
• Community & Metrics: Active issues/PRs (e.g., setup guides). (Stars: Emerging; Forks: Moderate; Last commit: Recent.)
• Why Best: Flexible and benchmarked; integrates with LangChain ecosystem for custom workflows.

PaperBench of OpenAI

Paper2Code of https://github.com/HKUDS/DeepCode

Five main categories, each with specific tools mentioned in the Nature article:

1. Literature Review Tools:

   • Gemini Deep Research - Conducts in-depth searches over 30 minutes with text, figures, and proper citations
   • OpenAI Deep Research - Advanced active learning for comprehensive literature exploration
   • SciSpace - "Chat with PDF" function for deep dives into specific papers
   • Other tools mentioned: Claude, NotebookLM, PDF.ai
   • check academic literature using either Semantic Scholar or OpenAlex APIs

2. Hypothesis Generation:

   • Research Rabbit - Visualization tool that generates interconnected webs of research by topic, author, or methodology
   • Allen Institute Tools - Upcoming hypothesis generation products combining ideas across papers into novel concepts
   • Elicit - Research assistant for generating experiments based on hypotheses

3. Experimental Assistants:

   • CRESt - Copilot for Real-world Experimental Scientist - helps craft and run experiments with robotic integration
   • Scite - AI research assistant for experiment planning and literature analysis
   • ChatGPT for Troubleshooting - Helps brainstorm explanations and debug experimental issues

4. Code & Statistics Tools:

   • GitHub Copilot - AI pair programmer for coding data analysis and visualizations
   • Amazon CodeWhisperer (Amazon Q Developer) - Code generation with security scanning capabilities
   • Cursor - Editor that makes data analysis easier with visualization capabilities
   • CatalyzeX - Locates and shares existing code from published papers

5. Scientific Writing Aids:

   • Paperpal - Checks academic manuscripts against journal submission guidelines
   • Thesify - Multi-stop platform for students with focus on grant writing and paper formatting
   • SciSpace - Templates for research proposals, literature reviews, and abstracts
   • Coral AI/Quillbot/Whisper - Translation capabilities covering dozens of languages

AI tools are spotting errors in research papers: inside a growing movement, Nature, 2025

• https://www.nature.com/articles/d41586-025-00648-5