Here is how to explore more deeply the judgements made by a particular judge

To explore the judgements that were made by a particular judge more deeply, we can look at the decisions made by that judge, whose scripts they were comparing, and the probability that the decision was 'correct'.

After the judging is completed, go to the Run judging session section and then highlight the judge you want to look at by clicking on the box next to them. Then click on the Decisions for Judge button (the button with the three lines).

You will then see this page:

Here are the important aspects of this Judge Decisions page:

• The judge whose decisions are being examined - at the top, you will see the judge's email address - you can click on this to set up an email to them;
• Chosen - the 'winning' candidate's name - you can click on the name to view their script;
• Chosen Score - the Scaled Score for the 'winning' candidate;
• Not Chosen - the 'losing' candidate's name - you can click on the name to view their script;
• Not Chosen Score - the Scaled Score for the 'losing' candidate;
• Time Taken - the time taken for this one judgement;
• Probability - based on the two candidates' Scaled Scores, we can calculate the probability that this judgement was 'correct' based on other judges' decisions on the task. Or rather, the higher this probability, the more likely the decision was in agreement with the other judges.

We can order the judgements by clicking any of the headings of the Decisions table.