Why two rounds
A congress with 1,200 abstracts and 40 reviewers cannot give every abstract three full reviews. Many congresses therefore review in two rounds. In the first round, the screening, every abstract gets one or two quick checks with a short form: is it complete, is it in scope, is it worth a full review? Only the abstracts that pass go on to the second round, the full review, with the full rating form and more reviewers.
In run.events, each round is a review stage. A session is in the stage of the bucket it sits in, so each stage needs its own bucket.
Step 1: Prepare a bucket for each stage
New submissions always arrive in the bucket In Evaluation, which every event has from the start. Use it as the bucket of the screening.
For the full review, create a second bucket:
- Open Content And Speakers and, under Configure, choose Buckets.
- Choose Add new, name the bucket, for example "Full review", and choose a Bucket Type of the kind In Evaluation.
- Save.
Step 2: Build the two rating forms
Build a short rating form for the screening, for example one criterion "Suitable for full review" and a remark, and a full rating form for the full review. See How do I build a rating form with weighted criteria and instructions for reviewers?.
Step 3: Set up the screening as the first stage
Every presentation type has its first review stage from the start.
- Open Presentation Types, choose the presentation type, and open its tab Review stages.
- Choose Edit on the first stage.
- Enter the Stage name, "Screening".
- Under Buckets, choose In Evaluation, the bucket where new submissions arrive.
- Choose the screening form as Rating form.
- Set Minimum reviews, for example 1, and the Due date for assignments.
- Save the stage.
Step 4: Add the full review as the second stage
- On the same tab, choose Add review stage.
- Enter the Stage name, "Full review".
- Under Buckets, choose your full review bucket.
- Choose the full rating form as Rating form.
- Set Minimum reviews, for example 3, and the Due date for assignments.
- If the full review should be blind while the screening is not, set its Review visibility.
- Save the stage, then save the presentation type.
Step 5: Run the screening
Assign the screening reviewers to the abstracts in the screening bucket, for example with Auto-assign, and let them rate. See How do I assign abstracts to reviewers automatically, and set how many reviews each needs?.
Step 6: Move the best abstracts on with Advance by score
When the screening is done:
- On the Content Organizer, open the menu of the screening bucket and choose Advance by score. The same action is in the selection bar of the Sessions page.
- Enter Advance at or above: abstracts with at least this score move into the full review.
- If you want, enter Decline below and choose a Declined bucket: abstracts below this score are declined at once. Abstracts between the two scores stay in the screening bucket for a closer look.
- Decide whether abstracts that do not yet have the stage's minimum number of reviews are left out.
- Look at the preview, which lists in columns which abstracts advance, which stay and which are declined.
- Confirm.
Scores run from 0 to 100 with a rating form, and from 1 to 5 with Five icons.
Step 7: Run the full review
The abstracts in the full review bucket are now in the second stage, with its form and its number of reviews. Assign the reviewers again, for example with Auto-assign, which now uses the second stage's Minimum reviews.
Good to know: the reviews of the screening stay with the abstract. The decision committee can read both rounds.
Related articles