How do I prepare the decision meeting from the reviewers' scores?

Rank the abstracts by score, find the ones the reviewers disagree on, and give the committee its papers.

Article Image Alt

What the committee needs

The review is done, and the scientific committee meets to decide. To decide well in a few hours, the committee needs three things: the abstracts ranked by score, the abstracts where the reviewers disagree, and the reviews themselves for the cases they want to discuss. run.events prepares all three.

Step 1: Check that the review is complete

Open Content And Speakers and, under Report, choose Review. Reviews done of needed should be at or near 100 percent. If reviews are still missing, see How do I see which reviewers are behind?.

On the morning of the meeting, stop further rating so the scores do not move during the discussion: open the bucket in evaluation and tick Lock ratings.

Step 2: Rank the abstracts by score

  1. Open Sessions and filter on the bucket in evaluation.
  2. Above the list, choose Sort by, then Rating. The best-scored abstracts come first.

Each card shows the abstract's score, its number of reviews and its reviewers' recommendations.

The Sessions page sorted by Rating, highest first, with the score and the reviews on each card

Step 3: Narrow the list for each part of the meeting

Search offers filters made for the decision:

  • Score from and to: for example everything above 70 for the clear accepts, everything below 40 for the clear rejections, and the band between for discussion.
  • Recommended: the abstracts that reviewers recommended for a given outcome, for example "accept as poster".
  • Criterion and Round: the score on one criterion, or in one review stage.
  • Reviewers disagree: the abstracts whose reviewers' scores differ by at least the presentation type's Disagreement threshold. These deserve a closer look.

A common way to run the meeting: confirm the clear accepts and the clear rejections in one go, and spend the time on the band in between and on the abstracts where reviewers disagree.

The review filters appear when at least one presentation type is rated with a rating form. With Five icons only, sort by rating and read the icons on each card.

Step 4: See the abstracts that need attention

On the Review dashboard, Sessions that need attention lists the abstracts with too few reviews, overdue reviews or disagreeing reviewers, with their score and the reason. Go through them before the meeting.

Step 5: Give the committee its papers

  • The reviews of selected abstracts: tick the abstracts on Sessions, then choose Documents and Review summaries in the selection bar. run.events prepares one PDF per abstract, packs them in a zip file in the background, and mails you a link to download it.
  • All scores in one table: on the Review dashboard, choose Export evaluation report. You get an Excel file with one sheet of abstracts and one sheet of reviews, which the committee can sort and filter.

Step 6: Record the decisions

During or after the meeting, record the decisions by moving the abstracts into buckets of the type Accepted or Declined. Nothing is sent to the authors yet. See How do I accept and decline many abstracts at once, also as another presentation type?.

Good to know: for your board's report, the sessions report shows the counts by status and by bucket. There is no ready-made acceptance rate; work it out from these counts.

Related articles