Paper Production Guide - uchicago-bfi-gnlab/lab_manual GitHub Wiki
Note: this guide is adapted from the gslab guide here.
Whenever we get ready to circulate or submit a draft of a paper, the following tasks need to be done. These will usually be allocated among the RAs with one person in charge of supervising a given task and pulling together the results in a single deliverable. Each task has a default time estimate and number of RAs assigned to it. If it looks like the time it will take to complete a task is much longer than the time estimate, PN/PG should be consulted.
One RP will be assigned to supervise production. They should
- Create github issues for each task and add to the project board. Note that a task should be a unit of work and a person. For example, when we split FACT to be done across two RAs, we should split the issue ticket in two.
- Keep track of each task and its completion status. It may be useful to have a "main" paper production issue with a table that tracks each task, issue number, task name, task lead and assignees, status, and priority.
- DAILY TASK: Supervisor should send a daily update on Slack about the current status of paper production, and dependencies across tickets
If the paper is tracked via git
- Create a main
paper_productionbranch - Each production task should branch off the
paper_productionbranch using standard gnlab procedure. Naming followsissueXXX_tasknameconvention - After a task is completed, the issue should be moved for PI review on the project board, and after it is reviewed then the task branch
issueXXX_tasknameshould be merged back intopaper_productionbranch. - If you get a merge conflict and choose to resolve it in a text editor, it's really important to be careful with what text editor you use. Some editors delete trailing white spaces which will wreak havoc on the formatting in lyx. We suggest using Visual Studio Code.
- If you accidentally do delete trailing white spaces, it's helpful but not sufficient to use the diff in Github to see where errors might have been introduced. You should also look at the spacing around citation/exhibit references, sticky notes, quotation marks, italicized words, and math.
-
Be a problem solver
- Most of the tasks are written around identifying problems. However, when you do identify a problem, please also propose a solution if possible.
- When working on a task, if you discover issues relevant to a different task, raise and log it in the github issue for that task.
-
Allocate your and our attention thoughtfully
- For any task where there are clear typos/ errors to be made that you are 100% confident we would want to correct, coordinate and implement these comments without PI review
- RAs should only spend time making the final deliverable that PN/PG will review clear and concise. Documents circulated only among RAs should not be made "pretty."
- Attach deliverables (comments or PDFs) to the relevant GitHub task and notify PG/PN with an @ reference. If working on a tight deadline, please notify through slack or e-mail.
- Scope References below to “the paper” apply also to the online appendix and any other document that we are planning to circulate externally, unless otherwise noted.
- An entire round of production should take no more than one week of work for 3-4 RAs.
- Tasks below are categorized as "AI Amenable", "Human centric", or "Mixed".
- For "AI Amenable" tasks, our default position is that the AI can reliably do this task, but you should still spot check output
- For "Human centric" tasks, as of June 2026, we found that heavy reliance on AI was more effort that having a human be the primary person doing the work.
- It is useful to give AI the .tex document, rather than the pdf because then it has access to the underlying table and figure labels, which makes referring to exhibits much easier.
Work Allocation
TBD
Tasks
- Turn on package management software (
checkpointfor R, TBD for Python) and ensure that results are unchanged. - Do this task before PRELIM and EXT
Work Allocation
TBD
Tasks
- Make sure STABLE task is done
- Bootstraps and simulations run with sufficiently high number of draws
- Quadrature accuracy set sufficiently high
- Tolerances on solvers satisfactory
- Exit flags for solvers indicate convergence
- Ensure there are no xxx, ccc, ggg, nnn, or track changes remaining in the draft, editor letter, or referee response letters
Work Allocation
RA to propose based on code base
Goal
- codebase: confirm that results reported in paper would be unchanged if all externals calls to Google Drive, Dropbox, etc. are pointed to the most recent version
- paper: update all the paths in Lyx from
input/needs_pathto to their production versions inout. compare old compiled PDF and new compiled PDF side-by-side to be sure that plots didn't change. Note that Material produced for referee letters only by screen-capping figures in other people’s papers or figures in our other research projects can stay ininput - Delete
needs_pathdirectory and its contents
Deliverables
- A list of any external calls that are out of date and would impact the results.
- A list of any source/targets specified incorrectly and any directories that need to be re-run.
- A revised paper.lyx where the figure and table paths are updated
Work Allocation
| Expected Hours | 50% Confidence Interval | # of RAs |
|---|---|---|
| 1 | (0.5 - 2) | 1 |
Goals
- All citations in the text match references in the bibliography.
- All references in the bibliography are cited somewhere in the text.
- Author names and years in text citations are correct and in-text citations follow the formatting guidance in the writing style guide including punctuation.
- If we are citing working papers, check the publication status. If they have been published update the reference accordingly.
- The references list is correct and uses a consistent style. (It is not important which style guide we follow, just that we are consistent.)
Notes/Standards/Useful Information
- Follow the citation guide.
- Working papers should be cited as:
- JMP/non-NBER: Authors. (year). Title. Working Paper
- NBER: Authors. (year). Title. Working Paper 12345, National Bureau of Economic Research
Suggested Workflow
- Give AI the pdf and the .tex file and instruct it to:
- Verify that there are no broken in-text citations
- Confirm all cited papers exist
- Ensure citations are consistently and correctly formatted
- Verify all citations are up to date with the current version of the paper
- Make sure to use an AI equipped with the necessary tools for web access (e.g. Claude Opus 5).
- With track changes enabled in Overleaf, correct small errors, such as an outdated working paper, missing page numbers, incorrect capitalization, etc. Note the errors and corrections in your message to the PIs.
- Flag larger mistakes that have no clear solution to the PIs—for example, a citation for a paper that doesn't exist.
- Make sure to recompile the pdf after making these changes to ensure th e edits didn't introduce broken in-text citations. Share a pdf version of the updated paper with the AI to confirm.
Deliverables
- A GitHub comment that records any changes that are substantive enough to require PI review. If you weren't able to propose a fix, this should also be included.
- An updated version of .tex and .bib files in overleaf with corrections.
Work Allocation
- Up to 16 hours (1 Academic and Research Specialist)
Input from gnlab
- A single pdf with entire paper including tables, figures, and appendices
- A lyx document with the input text.
Goals
- Follow rules outlined in writing style guide
- Typos, spelling/grammar errors, unclear wording, etc. corrected (don’t forget to include table notes, figure notes, footnotes, axis labels, appendices, etc.)
- Check that when a statistic in the text reports a number from a table it matches what is actually in the table
- Check that table/figure references refer to the correct table/figure, and that the table/figure presents the promised information (as far as you are able)
- Check that figures and tables are referenced in order
- Variable names, notation, and other concepts are used consistently; the same notation is never used to refer to two different things (this might be tricky if the notation isn't natural to follow for you, in which case skip this step)
- Suggest any other improvements to the writing you might have
Deliverables
Note: Use track changes in lyx
- A single lyx document with tracked changes. Any notes should be in yellow stickies, with your initials to GGG/NNN at the beginning, i.e. “[your initials] to GGG/NNN: I recommend a different word choice here for xx and yy reason”
- A single pdf document with any changes/comments that can’t be inputted in lyx clearly marked using Adobe Acrobat’s commenting tools.
- Upload both to github issue by dragging and dropping.
remarks
- If you make the same edit in several places (e.g. U.S., semicolons, etc), it is fine to just have a sticky explaining this the first time. This should helpfully save you time.
- First review the PDF, then review the Lyx. However, please enter edits/comments preferentially in the Lyx document since it is a bit easier for us to address issues efficiently from the Lyx doc
Work Allocation
- Up to 16 hours (1 RA)
Goals
- Correct spelling, grammar, figure/table refs, etc. Follow the draft style guide
- Note: If there is a recurring issue, inform PIs immediately via comment and @ tag below, and provide examples. PI will decide how to handle. Possible solutions: (i) address the issue throughout; (ii) ignore; (iii) do not change but flag all instances in the final deliverable. PN/PG will also instruct on whether the Draft Style Guide section of the RA manual should be updated to clarify the issue.
- This task does not extend to (if extant) the cover letter and referee replies.
Tasks (identical to ARS)
Note: For substantive or meaning changes, use track changes in Oveleaf, for small typo fixes, directly edit. Use AI to do the following:
- Check text (including table/figure notes, footnotes, axis labels, appendices, etc.) for typos, spelling/grammar errors, etc.
- Check that when a statistic in the text reports a number from a table it matches what is actually in the table.
- Check that figures and tables are referenced in order
Use AI to aid the following:
- Check that table/figure references refer to the correct table/figure, and that the table/figure presents the promised information (Note: Review to make sure each table reference is actually refering to the correct table, use AI as a second set of eyes)
- Variable names, notation, and other concepts are used consistently. The same mathematical notation is never used to refer to two different things. (Note: Review AI output and assess whether the same symbol in different sections refers to the same object or different objects)
Manually do the following (These tasks will take the majority of the time):
- Check text for unclear wording. The subject and verb of each sentence should be clear.
- Titles of sections, tables, figures, etc. are clear, descriptive
- Suggest any other improvements to the writing you might have
Tasks (unique to RA)
- Use AI to spellcheck (overleaf spellcheck isn't reliable)
- Within text, check that vertical white space looks reasonable. If you see a problem adjacent to a section header right-click on each header and change the spacing to 1.5 until the issue is resolved. If you see a problem elsewhere, use custom vertical space (insert -> formatting -> vertical space -> custom) to adjust.
Deliverables
- Tracked changes on Overleaf.
- Comments in Overleaf document. Any notes should be in yellow stickies, with your initials to GGG/NNN at the beginning, i.e. “%[your initials] to GGG/NNN: I recommend a different word choice here for xx and yy reason”
Work Allocation
- 6 hours each for initial pass (2 RAs)
- 2 hours to address PI feedback after first pass
- Note: There should be separate (duplicate) tickets for each RA working on this
Goals
- Every quantitative/factual claim is supported be one of (i) Table entry (ii) Figure or (iii) Citation
- Note: For external citations, please save a copy of the referenced paper as a pdf in
paper_slides/facts/directory. Use(authors)_(yyyymmdd)_(Journal abbrev. Or WP)_(paper_title)format. The sourcing statement/comment should reference the specific table/figure in the saved document that contains the fact.
Definition of a fact
A fact is:
- A direct quantitative statement. e.g “The parameter x is ”
- A statement of quantitative comparison. e.g “Model x has a better fit than model y”
- A statement that claims a date. e.g “The program started after September ”
Examples of qualitative statements that do not need to be checked
- “Becker and Friedman (1950) examine the role of markets for organs”
- “The model fits well”
Tasks
- Check that all facts are supported
- Most facts should already have a comment in the .tex document with a pointer to a source (i.e., table/figure either internal or external). RA should check that the sourced content actually contains fact.
- If they cannot find a source, mark the fact as UNSUPPORTED in a comment.
- If the source conflicts, add MAYBE WRONG to the comment, followed by an explanation of the problem.
- Process note for facts documented with "test_that" statements in code: The comment should refer to the name of the code file and include a snippet of the test_that statement so PIs and RAs can easily check the latest version of the file to confirm that the statement is correct. Example: "SOURCE test_that statement in representativeness_v2.R reading "xxx xxx""
Suggested Workflow
- Give AI a pdf version of the paper and have it create a markdown file that lists all the paper's facts, noting what section the fact occurs in and the sentence that it comes from. In the prompt, provide it with the definition of a fact that is described above. Make sure you tell it to include facts even if it is unsure about whether they meet the definition.
- With this markdown file, go through each fact in the paper and confirm that it is correct and supported with a source. If it is not correct, change it to be correct. If it is not supported with a source, find the source and include the source as a comment. For any facts that you can't find a source for, leave a comment that says "UNSUPPORTED".
- Once this is finished, give AI the .tex file and have it compile a list of all facts which do not have a source in the comments. If it finds any such facts, add a source if you can.
Notes
- Ensure that adding the in-line comments does not introduce errors to the paper's spacing
- The above note does not preclude making additional comments/raising notifications in the github issue
- After you submit the deliverable, expect to iterate on the factcheck portion until all the unsupported or maybe wrong claims have been addressed and all track changes have been accepted or rejected
Deliverables
- An updated version of the paper in overleaf where all facts have a source, are marked UNSUPPORTED, or marked MAYBE WRONG.
- A list of all UNSUPPORTED or MAYBE WRONG facts.
Paper_production DEF - Check sample definitions, variable definitions, empirical specification, model specification [Mixed]
Work Allocation
- 8 hours (1 RA) but review the scope of this task and optimal time allocation with PG/PN before starting.
Goals
- Main statements made in paper are consistent with code. This applies to sample definitions, variable definitions, empirical specification, and model specification (depending on which section of the paper you are working on). It can be helpful to read through an updated version of the paper before starting as it can be hard to track all changes since you last read a draft.
- The RA with the best knowledge of the codebase should do this task.
- Checking every variable and sample definition for some projects takes a large amount of time and is highly variable depending on the codebase that is being reviewed. Unless specifically directed otherwise, you should focus on checking the definition of the main sample(s) in the paper and variables in the core specification(s), and either ignore or just spot-check robustness analyses, supplemental analyses in appendices, etc.
- Do the task a second time independently with AI.
AI-Use
- Working with an agent is much more helpful for this task than using a chatbot as the task directly requires comparing the code and the paper
- Agents are fairly strong with this task, but they are inconsistent (may point to an error in one sentence but not a similar error in a subsequent sentence) and don't find everything. Additionally, they might flag points that aren't a concern. Use AI for one pass, but do not fully rely on it.
Deliverables
- A list of any inconsistencies between text and code
Paper_production AGGREGATION_AND_CR_IN_FIREWALL - Check aggregation standards and code review for Chase [Mixed]
Work Allocation
- 8 hours (1 RA, inside Chase)
AI use
- AI is great using the paper
.tex-file to create a list of exhibits, exhibit names, and underlying source files as well as connecting it to the relevant code files and sections. - If CSVs include sample counts (minimum cell sizes) then AI can include them in the CSV as well but these should be checked.
- Methods as well as minimum cell sizes for certain operations (especially pseudo-medians) need to be throughly checked by a human. When minimum cell sizes come from stats_for_text code, it should be checked whether the samples actually align with the code that created the exhibits.
Goals
- Aggregation standards document filled out. Details inside Chase. Must be turned in with pdf of paper for any PUBLIC disclosure review
- Any required code review documents also filled out. This document usually has one row for each pull request that has been approved Note: see the confluence page "JPMCI Release Workflow" for details. Eventually we need to submit every item listed in "Initial Handoff".
Deliverables
- An excel file with the aggreation standards documented. File should be (a) emailed to PIs at Chase email and (b) saved in the project's repo
- Any required code review documents
Work Allocation
XXX
Goals
- formatting for social media of the paper's key plots following guidelines here
Deliverables
-
pngfiles
Work Allocation
- 4 hours (1 RA)
Goals
- Funding sources are acknowledged.
- Seminar participants and those who provided comments are acknowledged.
Notes
- The “meetings” folder on google drive will list comments from seminars, e-mails and conversations.
- Getting a list of funding sources is harder but a good place to start is acknowledgments on other recent papers. You can also ask PN/PG if NSF or other ongoing funding sources should be added.
Deliverables
- A list of unacknowledged sources of funding and comments.
Work Allocation
- 2 hours (1 RA)
Goals
- Floats (figures and tables) are in the correct order
Tasks
- Check that all tables/figure are referenced at least once in main text or appendix.
- Check that all content in the appendix is referenced at least once in the paper
- Within each section (tables, figures, discussion) of the online appendix, check that content appears in the order it is referenced in the paper.
- Exception to the above: any figure/table using Chase data that is in a referee or editor response (when applicable) but NOT in the regular paper still needs to be included at the end of the appendix to meet disclosure requirements. These figures/tables do not need to be referenced in the text.
Deliverables
- Produce a list of plots that are not referenced
- Produce a list of plots that are referenced in the wrong order, and a suggestion of where they should be
Work Allocation
- 16 hours (1 RA)
Goals
- Color plots print well in black and white;
- markers are sufficiently distinct
- Axis labels, row and column headers in tables, etc. should only have the first word capitalized
- Axis labels, subtitles, facet labels, and legends accurately describe plot contents.
- Plots follow the guidelines in gslab Data Visualization - this in turn links to Schwabish (2014)
Deliverables
- Produce a list of plots that have issues in B&W, and propose fixes.
- Produce a list of plots that you think have legibility issues, fail to follow good guidelines, or are otherwise not acceptable
- Modified plots which incorporate suggestions. These can either be put directly into the paper after a short loop about the suggestions, or added to a comment in the issue thread for unreviewed suggestions.
Work Allocation
- 8 hours each (2 RAs)
- Note: There should be separate (duplicate) tickets for each RA working on this
Goals
- All nontrivial mathematical claims in the paper are documented.
- By default, this does not include checking statements within proofs. However, PN/PG should be consulted at the beginning of this task to confirm the desired scope.
Deliverables
- A list of all theoretical claims that are made in the text of the paper that are not supported by
- (i) proofs in the main appendix
- (ii) discussion/proofs in the online appendix
- (iii) discussion/proofs in text.pdf. E.g., we may say It is easy to show that equations A, B, and C together imply equation D. You should not include claims that are completely obvious. We’re looking for things where if somebody came back and said I don’t believe this is true we would need to go back and do at least a couple of lines of algebra to confirm that we’re right
- A version of claims.pdf with comments noting, for each claim, either
- (i) a place it is referenced in the paper or online appendix, or
- (ii) that the claim does not appear to be referenced
task for a PI
- upload as NBER working paper
- send to BFI as working paper
Work Allocation
- 4 hours (1 RA)
Goals
- All figures and tables in the slide deck are up to date
Deliverables
- A revised slide deck with figures and tables as they appear in the paper
- A list of places where the interpretation of a figure or table appears to have changed (e.g. such that the sentence we would say in the voiceover is different)
This task done by an RA with Chase data access
Goals
- Clarify what code corresponds to the released document
- Clean up and allocate ownership correctly for data tables
Deliverables
- "tag" the final commit (bitbucket instructions here)
- Verify that the
README.mdfile includes all (current and historical) builds we definitely want to preserve. This will help us if data tables are getting deleted periodically (which is not the case at the moment). - Transfer ownership to Peter or Pascal and verify by checking their SID
This task is often done more than once because some code lives in various restricted environments. AI can usually do this well, if it has a good starting point.
Deliverables
- Update
README.mdfollowing guidelines here - Update date on
README.md - "tag" the final commit (github instructions here)
This task is only done for AEA journals
Work Allocation
- 4 hours (1 RA)
Goals
- All data sources are cited following AEA (guidelines)[https://www.aeaweb.org/journals/policies/sample-references]
Deliverables
- Add missing dataset citations using Zotero
- A version of
paper.lyxwhere these data sources are included in the bibliography. The citations do not need to be added to the text. - A github comment listing the JPMCI data sources which are missing from appendix table
Work Allocation
- 6 hours each (2 RAs)
- Note: There should be separate (duplicate) tickets for each RA working on this
Goals
- Every question or comment by an editor or referee is addressed directly in the corresponding response letter.
- Every statement made in a cover letter to the editor or a reply to a referee is correct. If the statement refers to a change to the paper, the paper has changed as indicated since the previous submission. If the statement is a table or figure referenced but not shown in the paper, the table or figure presented to the editor/referee matches a supporting document.
- If a table or figure is included in the referee report and uses Chase data, it is also included in the paper. Even if it is not referenced in the paper, it should be included at the end of the online appendix.
- Conduct PROOF on the cover letter and replies, with a focus on clear errors or issues that would cause confusion. Consistency in style is not that key in replies since they will not be published and are intrinsically transient.
- Fill in correct exhibit numbers in paper when listed as xxx.
- Check all figure and table numbers in referee reports to make sure they are correct.
- Ensure that the response letter faithfully reproduces the editor and referee letters (e.g. there should be no missing paragraphs, that aren't addressed).
Suggested Workflow
- Conduct PROOF on the letter. While doing this, in addition to proposing edits to the writing for clarity, also note whether the editors' and referees' comments have been fully addressed. Suggest edits or new material if you believe they have not been.
- Give AI the the .tex file and pdf of the response letter. Have it create a list of all the typos, grammatical errors, unclear sentences, inconsistencies, and factual errors in the letter. Make sure that these files already have the changes you proposed as part of the first step so that nothing is double counted.
- Along with the response letter, give AI the editor and referee letters and have to confirm that the response letter faithfully reproduces these letters (i.e. no misquotes or missing paragraphs that aren't responded to).
- Give AI the tex file of the paper itself. Have it confirm that all references to exhibits in the response letter are correct and that the response letter accurately represents the paper.
Deliverables - An updated response letter tex file in overleaf with in-line comments and proposed edits in track changes.
Paper_production QUOTES- Update quotes in editor and referee letters match the most recent version of the paper (AI Amenable)
Work Allocation
- 4 hours (1 RA)
Goals
- All quotes from the paper text in editor and referee letters reflect the most recent version of the paper as it appears after all proofreading has been completed
Deliverables
- A tracked changes version of editor, R1, R2, R3, and R4 letters (if they exist)
Work Allocation
- Consult PG/PN
Goals
- We have flagged all errors in translating our submitted manuscript into the journal’s typeset format.
- Note: the goal here is not to proofread the paper (i.e., not to perform the above-listed steps). We presume we will have done that as of the last submission, so the only possible remaining errors are those from typesetting.
Deliverables
- A single pdf document that lists all "meaningful" discrepancies between our original typeset manuscript as of the last submission to the journal and the galley proofs. Construct this list by using Adobe Acrobat’s commenting tools to markup the galley proof.
- Pay special attention to the formatting of tables, figures, and equations, as these are where most discrepancies tend to arise.
- "meaningful" is tricky to define. Exclude recurring stylistic edits to make the writing consistent with the journal's style guide (e.g. removal of hyphens, changes in capitalization, change from "panel (a)" to "left panel)). Sometimes, this is a judgment call. If you aren't sure if the change is meaningful, do tell us about it.
- In the same pdf document, also comment on anything else that looks like a typo or error that you happen to come across. Note that you should be looking only for discrepancies with respect to the last submission, not doing any other form of proofreading. But if you do notice a probable error along the way it is best to flag it as it may still be possible to correct it.