As researchers we would normally export the raw decision data and process it ourselves prior to conducting our analyses.

Exporting results

Go to Run Judging Session and click Decisions download a CSV file of the raw decision data.

The CSV file will look something like the following:

Each row in the table is a single pairwise comparison.

• judgeName - The email address of the expert who made the comparison.
• timeTaken - The length of the judgement in milliseconds.
• chosen - The Code of the chosen (winning) artefact.
• notChosen - The Code of the losing artefact.
  • You can link this to the artefact PDF filename by exporting the Candidate results CSV file.
• createdAt - The date and time the judgement was made.

Processing results

The Decisions CSV file can be transformed into artefact scores for research purposes by installing the nmmBTm package for R.

############ R EXAMPLE CODE ############
#To fit the BTm model#
require(nmmBtm)
#> Loading required package: nmmBtm
decisions <- read.csv(file.choose(), header=T)
#> get the decisions
data("decisions")
mdl <- nmmBtm::btmModel(decisions)
#write the artefact scores to a CSV file
artefactscores <- mdl$effects
write.csv(scores,"artefactscores.csv")

We do not go into detail in this guide about the underlying model, but start with this PDF for further details.

This outputs a CSV file called artefactscores as follows.

The columns are: