Metadata and author page corrections - acl-org/acl-anthology GitHub Wiki

This page describes how to respond to issues that request changes to (a) metadata for a paper/volume or (b) one or more author pages. Scripts to support these tasks live in bin/correct.

Processes for ingesting new data, updating PDFs/attachments, and creating DOIs are described elsewhere.

A metadata correction is requested by opening an issue—one issue per paper—using the "Fix data" button to launch the dialog that fills in an issue template with JSON data. Metadata corrections then follow this issue template.

Corrections may be requested for the following three items: title, author, and abstract. In each case, they are reviewed manually. We have a hard rule that all approved metadata corrections must reflect the same information as the PDF found in the anthology. If the requested changes diverge from the PDF, the requester will be asked to submit a PDF correction to be completed before approval of the new metadata. In such a case, the issue should be labeled as waiting. In cases where the reviewer of the correction believes the intent of an author requesting a correction is unclear, they should leave a comment asking for clarification and label the issue as waiting.

Title corrections are generally quite simple. There are only a few things we check for:

• Wording
• Spelling
• Fixed-case words (e.g., LLM)

After ensuring the title appears the same as in the PDF, it can be labeled approved.

Author corrections can either take the form of a correction to the appearance of some name(s) or the ordering of the names. As usual, we do our best to ensure that the correction is accurately reflecting the PDF. The items we check for are:

• All names in the PDF appear in the metadata
• All names are spelled correctly (including diacritics)
• Forenames and surnames are separated properly
• Single-name author names fall in the last field
• Names are ordered in the same manner as the PDF

To fix these items, the reviewer should edit the first, last, and authors_new fields. The id field should not be modified: it indicates the current ID so that the script can align entries between the old and new author lists.

In the case that the requester has missed some corrections, they should be added manually to conform with the appearance of the PDF. Once the metadata is set to match the PDF, the issue should be labeled approved.

We also occasionally get authors who request author page corrections through comments on their created issue. We direct those people to our written instructions and follow the process outlined in the section below.

These are generally the most time-consuming changes to review as we need to check both the text and the formatting. To ensure the text matches the PDF, we check the difference between the requested text and the text in the PDF (e.g., using Diffchecker). We then verify side-by-side that any formatting found in the PDF that our system allows for is represented in the metadata correction. These include:

• LaTeX <tex-math>...</tex-math>
• Bold <b>...</b>
• Italics: <i>...</i>
• Underline: <u>...</u>
• Small caps: <sc>...</sc>
• Typewriter text: <tt>...</tt>
• Paragraph breaks: <par/>

We also allow abstracts to contain:

• URLs: <url>...</url>
• Hyperlinks: <a href=...>

For these, we generally follow the requested correction. If the correction request does not include the URL or hyperlink, we respect that omission as it's usually coming from an author of the paper, and they might have reasons for not wanting the inclusion of a link (e.g., avoiding naïve scraping). In cases where the request is not coming from the author, we default to matching the PDF exactly. Once the abstract metadata is set to match the PDF, the issue should be labeled approved.

An author page correction is requested via this issue template.

When processing these issues, we divide them into two main categories: simple verifications, which are easily automated, and other issues, which require more manual specification.

These are issues where all of the following hold:

• An author page presents as unverified (URL ends in /unverified/).
• All papers listed on this page belong to the same individual.
• This individual does not correspond to any verified person in the database.
• The request provides an ORCID/other info to associate with a new verified author page.
• There is no request for another action, such as a name change or moving papers from another page.

These issues should be manually reviewed by checking/editing to ensure that:

• The issue title conforms to the template "Author Page: name-slug/unverified"
• The URL of the unverified page is listed under Author Pages in the Anthology (OK if "/unverified" is missing from the URL as long as the provided URL redirects to the unverified page)
• An ORCID URL like https://orcid.org/0009-0003-2921-829X is provided.
  • NOT a URL with "myorcid" in it.
  • Check the URL to make sure it is the right person.
• The highest degree institution is provided and looks consistent with information available from OpenReview/ORCID profiles.
  • The institution should be a simple university name, omitting institute or department names.
  • Note that we are looking for the PhD institution (or highest degree below that if the author is not yet in a PhD program). Some authors (e.g., faculty) provide their current institution by mistake.
• Correction type checkboxes: Only "Verification" is checked.
• OpenReview profile URL is provided unless the author really doesn't have such a profile. Check a few URLs based on the name (https://openreview.net/profile?id=~Joe_Schmoe1, https://openreview.net/profile?id=~Joe_Schmoe2, ...) to confirm that the profile really does not exist.
  • Check that the ORCID URL is linked from the OpenReview profile. If this is missing, leave a comment in the issue asking for it (but don't wait to approve the request). If present, it should match the ORCID in the issue.

In addition, click through to the author page and perform sanity checks on the papers:

• Do they really all belong to the same author?
  • Usually institutions are a good cue; sometimes coauthors as well.
  • If it is a very prolific author I wouldn't necessarily look at every single paper, but look at some newer and older papers.
  • Sometimes the issue creator assumes you can disambiguate them from others by affiliations. Ask for clarification if the name looks ambiguous and they haven't specified which papers are theirs.
• Do the papers have a correct author list? Opening up the Fix data dialog is a good way to preview and check that the author list matches the PDF.
  • Open a metadata issue if there are any errors with a paper.
  • If a metadata issue corrects something about the author in question, link it as a child of the author page issue. (Right sidebar > Relationships > Add parent)

If the author page issue is complete, label it as approved.

The script bulk_process_simple_verifications.py will loop over the approved simple verification issues and create a commit for each.

• If an issue title does not end in "/unverified", the ORCID URL is not in the correct format, or the checkboxes do not identify it as a simple issue, it will be skipped.

Push the branch with commits (not master) to GitHub and open a PR for it. You can combine metadata and author commits in the same PR.

In the PR, request a review from @nschneid or @mjpost.

Non-simple issues should be assigned to @nschneid, who will subsequently assign one or more of the labels:

• meta:dnm (disable name matching for a verified author: only keep explicitly linked papers on their page and move others to an unverified page)
• meta:move (move specific papers to a currently verified author page)
• meta:merge (multiple author pages have papers that should all be consolidated under a single author, who is already verified)
• meta:vermove (verify the author and assign specific papers to their page)
• meta:vermerge (verify the author and combine with another author page)
• meta:canonical (change something about names listed on the author page)

Currently, we do not scrape non-simple issues; we manually run scripts as necessary to address each issue. The main script is bin/correct/verify_author.py, which takes an ORCID and maps it to a set of papers (indicated by one or more existing author IDs, or a list of paper IDs), updating the people database as necessary (e.g. to create a new person or merge multiple persons). Other scripts assist with rarer scenarios encountered in this process, like the need to change an existing verified author ID, or to unlink an explicit author ID from a paper.

The issue will be labeled approved when its contents have been confirmed and pending once implemented in a branch/PR.

For recent OpenReview-based proceedings (reliably from 2025, and for a subset of venues in 2022–2024), we have author profile IDs in the XML. bin/correct/refresh_or_orcids.py queries openreview.org to look for ORCID links for any of these authors who do not already have an ORCID in our database. (Currently this is >20k authors.) Once the author adds an ORCID link to their OpenReview profile, we can therefore automatically create a verified person and link it explicitly to papers from many recent venues.

Notes about the script:

• Each run will create up to 500 new person entries. This means that large imports will be batched. Once data from the last batch is correct, rerun for the next batch.
• Take care to review warnings/errors logged by the script. These include
  • New ORCID-verified persons that resemble legacy-verified persons but cannot be positively linked with OpenReview data (can be merged with verify_author.py)
  • Conflicts when a current verified author includes explicit papers from distinct OpenReview profiles with conflicting ORCIDs (usually this means disambiguation is necessary)
  • Automatic merges of an existing legacy-verified author and an existing ORCID-verified author who share an OpenReview ID.

Currently the script finds 10 invalid ORCID links in OpenReview, which we simply ignore.

Note that the script that queries openreview.org obtains ORCIDs only; it does not consult institutional or publication metadata in OpenReview profiles.