How do I assign abstracts to reviewers automatically, and set how many reviews each needs?

Set how many reviews each abstract needs and how many each reviewer takes, then let Auto-assign propose the reviewers and check the proposal before you confirm.

Article Image Alt

What Auto-assign does

Assigning a thousand abstracts by hand is not realistic. Auto-assign proposes reviewers for many abstracts at once. For each abstract, it prefers reviewers whose topics and presentation types match, skips reviewers in conflict and reviewers who have reached their maximum, and gives each abstract as many reviews as it needs. You see the whole proposal before anything is assigned.

Step 1: Set how many reviews each abstract needs

The number of reviews is set on the review stage of the presentation type. Every presentation type has its first stage from the start.

  1. Open Presentation Types, choose the presentation type, and open its tab Review stages.
  2. Choose Edit on the first stage.
  3. Check that Buckets names the bucket in evaluation where the abstracts sit.
  4. Set Minimum reviews, for example 3: an abstract is fully reviewed once it has three reviews. Set Maximum reviews if you want a ceiling.
  5. Set Due date for assignments, the date by which reviews are due.
  6. Save.

Without Minimum reviews on the stage, Auto-assign proposes nobody for its abstracts.

Good to know: there are two pairs of minimum and maximum, and they are easy to confuse. Minimum reviews and Maximum reviews on a stage are the reviews each abstract gets. Minimum reviews per reviewer and Maximum reviews per reviewer in step 2 are the abstracts each reviewer takes.

Step 2: Set how many abstracts each reviewer takes

  1. Open Call for Content Settings and its tab Review and limits.
  2. Set Minimum reviews per reviewer and Maximum reviews per reviewer for the whole committee.
  3. Save.

For one reviewer who should take more or fewer, open Content Team, choose the reviewer, and set their own Minimum reviews and Maximum reviews under Review limits. Choose Apply.

Set a maximum. Auto-assign looks first at how well a reviewer matches, and only then at how much work they already have. A well-matching reviewer with no maximum can be proposed for every matching abstract. A maximum spreads the work.

Step 3: Give the reviewers their topics

On Content Team, choose each reviewer and fill in Topic preferences: the labels and presentation types they know best. Choose Apply. Auto-assign proposes reviewers with matching topics first. Topic preferences only change the order of the proposal; they do not prevent other assignments.

Step 4: Run Auto-assign

  1. Open Sessions and filter on the bucket in evaluation.
  2. Tick the abstracts, or choose Select all.
  3. In the selection bar, choose Auto-assign.

The same action is in the bucket's menu on the Content Organizer.

Step 5: Check the proposal and confirm

The fly-out shows one card per abstract, with each proposed reviewer and the reason they were chosen, for example "Matches Oral presentation · Food allergy".

  1. Look through the cards. Where a reviewer does not fit, choose Swap on that reviewer and pick another from the list of reviewers who would fit.
  2. Check the Due date.
  3. Look at the abstracts listed as skipped, with the reason, for example no reviewer left without a conflict.
  4. Choose Assign.

The Auto-assign fly-out, with one card per abstract and the reviewers proposed for it

Which abstracts are left out

Auto-assign skips an abstract when it is not in a bucket in evaluation, when the bucket is locked, when the stage sets no Minimum reviews, or when the abstract waits for a chair's changes to be approved. Assign these by hand, or fix the cause and run Auto-assign again on them.

Step 6: Tell your reviewers

run.events sends no mail to reviewers when abstracts are assigned to them. Write to your committee yourself and tell them that the review has started, where to find their abstracts (the card Reviewer, button My reviews) and the due date.

The due date marks late reviews as overdue on the Review dashboard. It does not close the review: reviewers can still rate after it. To stop all rating in a bucket, tick Lock ratings on the bucket.

Related articles