How are submissions reviewed? - acm-toce/documentation GitHub Wiki

Once a week, the Editor-in-Chief (EiC) conducts editing activities on the ACM TOCE Manuscript Central website's EiC-Admin tab, which provides a view of all submissions for which the EiC does not have a conflict of interest.

On a weekly basis, they will:

  • Identify any "Overdue Reviewer Response", "Overdue Referee Invitation", or "Awaiting Referee Selection" papers that are substantially late, and reach out to AEs if they need assistance in getting reviewers to review.
  • Review any "Awaiting Decision Approval" recommendations from Associate Editors (AEs)
    • Review the email drafted by the AE.
    • Review the reviews obtained by the AE.
    • Communicate with the AE via Manuscript Central if their recommendation or the reviews are out of compliance with the journal's reviewing guidelines or there are other concerns.
    • Once the AE's draft email is final, correct any typos and send the decision to the corresponding authors.
    • After the decision goes out, they will open the submission in Reciprocal Reviews, confirm the AE has marked every reviewer assignment complete, then press Mark done. This compensates the editors assigned to the submission and closes it — so it only pays the editor who claimed the paper; if nobody ever claimed it, claim it first. It is blocked while any approved assignment is still uncompensated, and it cannot be undone.
  • Review any pending compensation requests and proposed transactions at the venue, including mints proposed to cover compensation when the reserve runs short.

In the first week of each month, the EiC will also process new submissions. Only submissions that are complete, properly formatted, and anonymized before the 1st of the month (anywhere on Earth) are included in that month's cycle.

[!IMPORTANT]
The monthly cycle is for several reasons: 1) it gives a due date for authors to target, 2) it helps keep the editing workload more predictable for the EiC and Associate Editors (AEs), and 3) it sets a healthy pace for review assignment (which should take about 1 month), reviewing (which should take about 1 month), and recommendation (which should take less than a month). This pace creates a roughly 3-month cycle for a paper.

New submissions are processed by the EiC as follows:

  • All submissions are first reviewed by the journal admin for formatting and anonymity requirements. If they find problems with these, the submission will be "unsubmitted." Authors may resubmit whenever they are ready; a submission must be complete, properly formatted, and anonymized before the 1st of a month to be considered in that month's cycle.
  • The EiC reviews submissions for scope, identifying any out-of-scope desk rejects and drafting the rationale.
  • After submissions are reviewed by the admin, the EiC will:
    • Send out desk rejects (these may be sent as soon as they are identified, ahead of monthly AE assignment)
    • Identify all AEs who have fewer than 3 active assignments in the past 3 months and are not on leave, and sort them in decreasing order by number of current assignments.
    • Review each paper's cover letter and history of submission to gather any context about the submission.
    • Review the paper's automatically generated plagiarism check to see if issues need further investigation. If there is an issue, the EiC will research the overlap and communicate with authors or ACM as necessary.
    • Claim the submission in Reciprocal Reviews. New submissions appear there as needs an editor; press edit this to take it on. Because TOCE has more than one editor, nothing is assigned automatically — Reciprocal Reviews emails the editors instead, and whoever is handling the paper claims it. This is also how a conflicted paper reaches the right person: the Acting EiC claims it and the EiC does not.
    • Check the submission is fully paid in Reciprocal Reviews. Nothing in Manuscript Central blocks an unpaid paper, and a submission can sit part-paid while co-authors approve their shares. If it is not paid, they hold the assignment and write the corresponding author. For a revision, check the author has paid for it — a revision costs less than a new submission, but until it is paid for its re-reviewers cannot be compensated.
    • Choose the most expert AE from the eligible list to assign.

[!IMPORTANT]
Claiming is not optional bookkeeping. Until someone is editing a submission in Reciprocal Reviews, no assignment on it can be approved and it cannot be marked done — so the Associate Editor cannot compensate reviewers, and no editor can be compensated either. Claim the paper in the same sitting you assign the AE.

[!IMPORTANT]
The AE assignment order above ensures that all AEs are getting a similar workload but also not too overloaded at any given time, while maximizing expertise.

After an AE is assigned and accepts, they follow the process described in the Associate Editor guidelines.

Conflicts of interest

AEs are allowed to submit to the journal. Since this creates an obvious conflict of interest, the EiC ensures that AEs are never assigned to edit their own submissions and that they have no visibility into the confidential review process.

EiCs are also allowed to submit to the journal, as are authors who have a conflict of interest with the EiC (institutional, collaborative, personal). Any submission for which the EiC has a conflict of interest is handled by an Acting EiC, appointed by the EiC to handle all conflicted papers. The Acting EiC follows the submission process above and makes the final decision on the submission. The EiC has no visibility into the confidential review process, no influence over its outcomes, and receives only the same communication about the submission that other authors do.

[!IMPORTANT]
Editors are allowed to submit for several reasons: 1) they may have collaborators and doctoral students who need to be able to submit to ACM TOCE, 2) disallowing submissions would disincentivize research-active scholars from volunteering for the role. There are tradeoffs with allowing this, primarily perceptions of conflict of interest.