Automating LaTeX Proposal Writing with Open Source Multi Agent Frameworks - chunhualiao/public-docs GitHub Wiki

Automating LaTeX Proposal Writing with Open-Source Multi-Agent Frameworks

Short answer first If your goal is to automate research-heavy proposal drafting directly to LaTeX, the strongest open‑source multi‑agent building blocks today are:

  • Orchestrators: LangGraph for reliable, stateful multi‑agent control flows; AG2 (formerly AutoGen) or Microsoft AutoGen for group‑chat and swarms; CrewAI for a lean, Python‑native team‑of‑agents workflow
    • LangGraph1,
    • AG22, AutoGen3,
    • CrewAI4.
  • Research agents: GPT Researcher (multi‑agent, LangGraph‑based, long reports with citations), and LangChain’s Open Deep Research template; for PDF‑centric literature, add PaperQA2 (agentic RAG)
    • GPT Researcher 5,
    • Open Deep Research6,
    • PaperQA27.
  • Document ingestion and citations: Docling for robust PDF/DOCX parsing and ASR; GROBID for high‑accuracy reference parsing and BibTeX; Pandoc/Quarto for Markdown↔LaTeX; latexindent for auto‑formatting; Tectonic for CI‑friendly, self‑contained LaTeX compile
    • Docling8,
    • GROBID docs9,
    • Pandoc10,
    • Quarto PDF11,
    • latexindent12,
    • Tectonic13.

What “automating LaTeX proposal writing” actually involves A practical system covers: (1) scoping and compliance extraction from the CFP; (2) literature and preliminary data synthesis with citations; (3) outline and work‑plan generation; (4) drafting with consistent templates; (5) figure/table and budget narrative stubs; (6) bibliography management; (7) LaTeX formatting/compilation; and (8) iterative review (critic/reviewer agents). Multi‑agent frameworks are ideal because they let specialized agents hand off tasks (planner → researchers → writer → reviewer → compiler) with explicit state, routing and tool use LangGraph1, AutoGen docs14.

Best open‑source multi‑agent orchestrators for this job

  • LangGraph (LangChain): Models agents as graph nodes with static edges or dynamic handoffs; offers supervisor, network, hierarchical patterns; built for stateful workflows and persistence. It’s well documented with research/report examples, and integrates smoothly with search/RAG and tool calling—great for a proposal “pipeline” you can visualize and control LangGraph1.
  • AG2 (formerly AutoGen): Evolution of Microsoft AutoGen under open governance. Provides group chats, swarms, nested and sequential chats, human‑in‑the‑loop switches, and tool integration; Apache‑2.0 + MIT legacy licensing. Especially good when you want conversational teams that negotiate and critique sections before handoff to a writer/formatter AG22, AutoGen3.
  • CrewAI: Lean, Python‑native “Crews” (autonomous teams) and “Flows” (precise orchestration). Simple to stand up, friendly to productionization, and popular for content and report generation patterns where roles like Researcher, Writer, and Editor collaborate CrewAI4, CrewAI docs15.

Highly relevant research/writing agents and components

  • GPT Researcher: An autonomous deep‑research agent that can run as a LangGraph multi‑agent assistant (planner, crawler, summarizer, publisher) to produce 5–6 page, citation‑rich reports (PDF/DOCX/MD). It maintains memory, aggregates many sources, and is purpose‑built for accurate, unbiased research—ideal to generate Related Work and Background sections you can drop into LaTeX GPT Researcher5, GPT Researcher docs16.
  • Open Deep Research (LangChain): A configurable deep‑research agent integrated with multiple model providers and search APIs, runnable in LangGraph Studio; provides a clean starter if you want to own the research graph and instrumentation Open Deep Research6.
  • PaperQA2: Agentic RAG for scientific PDFs with grounded answers and in‑text citations, plus metadata‑aware embeddings and re‑ranking. Great for turning a folder of prior work and papers into precise, citeable content blocks PaperQA27.

Document processing, citations, and LaTeX toolchain (open‑source)

  • Parsing and citations:
    • Docling: parses PDFs, DOCX, PPTX, images, and audio with advanced PDF layout understanding—useful to mine previous proposals/figures or transcribe lab calls into text for method sections Docling8.
    • GROBID: industry‑grade reference/header/full‑text parsing with REST API and Docker; exports BibTeX and consolidates via CrossRef/biblio‑glutton; strong F1 metrics on reference parsing. Use it to normalize references and auto‑build your .bib GROBID docs9, GROBID GitHub17.
  • Authoring and compilation:
    • Pandoc/Quarto: write in Markdown and reliably emit LaTeX/PDF; Quarto uses Pandoc→LaTeX→PDF (xelatex by default) and can keep the intermediate .tex. This is powerful when your agents draft in Markdown and you want reproducible LaTeX outputs Quarto PDF11, Pandoc18.
    • Tectonic: modern, self‑contained LaTeX engine ideal for headless CI builds and agent‑driven compilation (GitHub Actions supported) Tectonic13.
    • latexindent: auto‑formats LaTeX source for clean diffs and enforcement—run it as a finalizer agent/tool in your pipeline latexindent12.
  • End‑to‑end manuscript workflow (optional):
    • Manubot: Git‑based, CI‑driven manuscripts with automated citations via Pandoc; useful if you prefer collaborative Markdown first, then produce .pdf/.docx; can coexist with a LaTeX template by keeping .tex via Quarto or Pandoc Manubot19, Manubot org20.

Three proven open‑source stacks (pick based on your preference)

Stack A (most controllable, research‑first): LangGraph + GPT Researcher + PaperQA2 + LaTeX toolchain

  • Orchestration: LangGraph supervisor orchestrates Planner → Web/Local Researchers (GPT Researcher nodes) → Summarizer → Writer → Reviewer → Compiler LangGraph1, GPT Researcher5.
  • Literature: PaperQA2 agent for folder(s) of PDFs and cached embeddings; feed grounded, citeable snippets to Writer PaperQA27.
  • Citations: GROBID microservice parses extracted references and exports BibTeX; integrate with your .bib GROBID docs9.
  • Drafting: Draft in Markdown sections; Pandoc/Quarto to LaTeX; keep-tex for inspection; Tectonic compiles PDF in CI; latexindent formats source Quarto PDF11, Tectonic13, latexindent12.

Why this stack: LangGraph gives programmatic control and persistence; GPT Researcher and PaperQA2 cover web+PDF research with citations; the Pandoc/Quarto→LaTeX route is robust and reproducible LangGraph1, GPT Researcher16, Quarto PDF11.

Stack B (lean, content‑team style): CrewAI + PaperQA2 + Pandoc/Tectonic

  • Orchestration: CrewAI crew with roles Researcher(s), Writer, Editor, and Compiler Flow; simpler to wire up than a custom graph CrewAI4.
  • Literature: PaperQA2 for local corpus; optional Docling to ingest prior proposals/DOCX; optional GROBID for BibTeX PaperQA27, Docling8, GROBID9.
  • Output: Writer emits Markdown per section; Flow converts to LaTeX (Pandoc) and compiles (Tectonic); latexindent formats Pandoc10, Tectonic13, latexindent12.

Why this stack: fastest to prototype; good for teams who want a “crew” metaphor and quick wins in proposal outline and narrative generation CrewAI15.

Stack C (conversation‑centric teams): AG2/AutoGen + GPT Researcher + GROBID + Tectonic

  • Orchestration: AG2 group chat with speaker selection or swarms (Planner, Debate between two Researchers, Critic, Writer, Reviewer, Compiler) AG22, AutoGen3.
  • Research: GPT Researcher as a specialized agent inserted into the swarm; good handoff to summary/writer GPT Researcher16.
  • Citations and compile: GROBID for references, Tectonic for builds GROBID docs9, Tectonic13.

Why this stack: if you prefer dialog‑based collaboration (debate/critique) among agents before drafting sections, AG2/AutoGen patterns shine AG22, AutoGen3.

Optional platform and alternatives

  • OpenAgents: full‑stack, web‑UI platform that can host multiple agent types (data, plugin, web). Useful if you want a shared UI to run your research and drafting agents with browser control and tool orchestration OpenAgents21.
  • MetaGPT: multi‑role SOP‑driven framework that outputs “documents” among other artifacts; more software‑engineering oriented but adaptable for structured deliverables MetaGPT22.
  • smolagents (Hugging Face): minimalist agent framework that “thinks in code,” supports multi‑agent hierarchies; good for embedding tool logic in Python code with sandboxing smolagents docs23, smolagents GitHub24.

Concrete agent roles that map to proposal work

  • Planner/PM: reads CFP, builds section plan, milestones, and review gates; routes tasks to agents via supervisor graph LangGraph1.
  • Web Researcher(s): GPT Researcher instances run scoped queries, aggregate sources, return citeable notes and images GPT Researcher5.
  • PDF Corpus Analyst: PaperQA2 over your prior work and domain papers for grounded answers/snippets with inline citations PaperQA27.
  • Citation Parser: GROBID service normalizes references/BibTeX; ensures duplicated DOIs are consolidated GROBID docs9.
  • Writer: converts outlines + evidence into LaTeX‑ready Markdown or raw .tex templates Quarto PDF11.
  • Reviewer/Critic: evaluates sections for clarity, compliance, risk, and alignment; can follow debate patterns (AG2/AutoGen) AutoGen3.
  • Compiler/Formatter: runs Pandoc/Quarto→LaTeX, Tectonic compile, latexindent; returns artifacts and logs Pandoc18, Tectonic13, latexindent12.

Licensing and maturity snapshot

  • LangGraph/LangChain: active ecosystem, production case studies; Apache‑2.0 for LangChain core components LangGraph1.
  • AG2/AutoGen: AG2 is Apache‑2.0 (modified code) with MIT legacy from AutoGen; AutoGen itself is MIT with active community AG22, AutoGen3.
  • CrewAI: MIT‑licensed, independent of LangChain; rapidly evolving with strong community CrewAI4.
  • GPT Researcher: Apache‑2.0, multi‑agent mode powered by LangGraph; exports PDF/DOCX/MD GPT Researcher5.
  • PaperQA2: Apache‑2.0; agentic RAG with strong claims on scientific tasks PaperQA27.
  • Docling: MIT; advanced multi‑format parsing (PDF/DOCX/Audio) Docling8.
  • GROBID: Apache‑2.0; widely adopted, REST+Docker; exports BibTeX GROBID docs9.
  • Pandoc: GPLv2+; universal converter (Markdown↔LaTeX↔PDF) Pandoc10.
  • Quarto: built atop Pandoc; robust PDF/LaTeX workflows Quarto PDF11.
  • Tectonic: modern LaTeX engine fit for CI Tectonic13.
  • latexindent: permissive, widely used formatter latexindent12.
  • Optional: Manubot (open, automated manuscripts) Manubot19, OpenAgents platform (Apache‑2.0) OpenAgents21.

Minimal implementation blueprint (Stack A example)

  • Ingest and research
    • Spin up LangGraph project with nodes: Planner → WebResearch (GPT Researcher) → PDFResearch (PaperQA2) → Summarizer → Writer LangGraph1, GPT Researcher5, PaperQA27.
    • For internal corpora, parse prior proposals/whitepapers with Docling; store texts for PaperQA2 Docling8.
  • Citations pipeline
    • Run GROBID service; send extracted refs or full PDFs’ reference sections; output BibTeX; dedupe and inject into a central .bib GROBID docs9.
  • Drafting and compile
    • Writer emits Markdown sections following your LaTeX template variables; Quarto/Pandoc converts to LaTeX; keep-tex for inspection; Tectonic compiles to PDF; latexindent formats .tex for review Quarto PDF11, Pandoc18, Tectonic13, latexindent12.
  • Review loop
    • Critic nodes score clarity/risks, request rewrites; optional AG2 debate step between reviewers before accepting a section AG22.

When to choose which

  • Need explicit control, reproducibility, and easy debugging: prefer LangGraph; it’s built for explicit state and handoffs you can visualize and test LangGraph1.
  • Want fastest path to a working “team” that drafts and compiles: choose CrewAI; wire roles and a simple Flow for compilation CrewAI15.
  • Like conversational teams with debate/critique: go AG2/AutoGen with group chat or swarm patterns AG22, AutoGen3.
  • Heavy on PDF science literature: add PaperQA2 regardless of orchestrator PaperQA27.
  • Strict LaTeX CI requirements: use Tectonic + latexindent in the finalization stage Tectonic13, latexindent12.

Common pitfalls and how these tools help

  • Hallucinated citations: ground with PaperQA2 and enforce source‑backed notes from GPT Researcher; validate refs via GROBID consolidation PaperQA27, GROBID docs9.
  • Messy LaTeX: separate authoring (Markdown) from typesetting; use Pandoc/Quarto → LaTeX and latexindent before Tectonic build Quarto PDF11, latexindent12.
  • Hard‑to‑debug agent loops: prefer LangGraph’s explicit graphs and supervisor patterns; add stop conditions LangGraph1.
  • CI failures: Tectonic action with cache and optional biber; deterministic builds Tectonic13.

Quick links to start

  • LangGraph multi‑agent concepts and tutorials LangGraph1.
  • GPT Researcher (multi‑agent assistant, exports PDF/DOCX/MD) GPT Researcher5.
  • PaperQA2 (agentic RAG for PDFs with citations) PaperQA27.
  • GROBID (REST+Docker; BibTeX export) GROBID docs9.
  • Quarto → LaTeX → PDF; keep-tex for review Quarto PDF11.
  • Tectonic LaTeX engine for headless builds Tectonic13.
  • CrewAI (crews/flows) CrewAI4.
  • AG2/AutoGen (swarms, group chat, nested chats) AG22, AutoGen3.

If you share a target funder (e.g., NSF, NIH, ERC) and your current authoring setup (pure LaTeX vs Markdown→LaTeX, private corpus size, CI environment), I can tailor one of these stacks into a drop‑in repo for your lab, including role prompts, section templates, .bib automation, and CI scripts.