This vignette describes how to get started using the R exams package to create randomized OLAT tests (or quizzes). Several tutorials are available on r-exams.org which also help beginners with getting used to the features of the R exams package.
The following tutorials might be of interest:
R/exams questions are written in either pure markdown (.md
), R/markdown (.Rmd
) or R/LaTeX using Sweave (.Rmd
). For demonstration purposes this tutorial is only using R/Markdown files, more information can be found in the [First Steps][rexams-firsteps] tutorial on r-exams.org.
To create a new question we write a new .Rmd
file and edit the content with a text editor, e.g., a file called iso3-code.Rmd
. The following output shows the content of this file (iso3-code.Rmd
) which contains a simple ‘static’ single choice R/exams question:
Question
========
What is the ISO3 country code for Austria.
Answerlist
----------
* AUT
* AUS
* USA
Solution
========
The ISO3 country code for Austria is **AUT**.
Answerlist
----------
* Correct answer.
* False, AUS stands for Australia.
* False, USA stands for the United States of America.
Meta-information
================
exname: ISO3 Country Code
extype: schoice
exsolution: 100
exshuffle: True
Each question consists of three sections, namely “Question”, “Solution”, and “Meta-information”.
extype: schoice
), exsolution
defines the correct answer. Three possible answers, the first one is correct (thus 100
). exshuffle: True
tells R/Exams to shuffle the possible questions (3 out of 3; random order).Once we have saved our file (iso3-code.Rmd
) we can test the question by calling:
# Renders the question as html
exams2html("iso3-code.Rmd")
This will open the html-rendered version of the question in your browser. As exshuffle
is turned on you will get a different randomization every time the command above is called. It is suggested to do this multiple times to see if the randomization works as expected.
The function exams_metainfo()
allows to check/test if the solution defined in the Meta-information section is specified correctly.
# Test solutionstring
exams_metainfo(exams2html("iso3-code.Rmd"))
## Loading required namespace: base64enc
##
## exam1
## 1. ISO3 Country Code: 2
The output shows the correct answer is “2” which can be compared with the rendered html output (not shown here) to see if it is correct.
Once one has checked that the randomization works, the texts are correct, and the answer is correct a stress-test for the question can be done by calling stresstest_exercise()
.
stresstest_exercise()
renders the exercise multiple types (100 times by default) and returns some information about the randomization.
# Stresstesting 'iso3-code.Rmd' (100 times)
stresstest_exercise("iso3-code.Rmd")
stress <-plot(stress)
The output shows the runtime to generate the randomized questions and the position of the correct solution. As we use exshuffle: True
with three possible answers this should be close to a uniform distribution between 1, 2, and 3.
R/exams questions can also be dynamic. The following example is the content of a second question stored in vector-subsetting.Rmd
.
```{r generate question}
# Create a random integer vector of length 5
x <- sample(1:100, 5)
# Ask for the following index between 1:5
idx <- sample(1:5, 1)
# Wrong answers
wrong <- sample((1:5)[-idx], 2)
# The correct answer:
res <- x[idx]
```
Question
========
Vector subsetting in _R_. We can access elements/values
in a vectur using vector subsetting by index. Imagine we
have the following vector `x`:
```{r}
print(x)
```
If we call `x[`r idx`]`, what is the resulting value?
Answerlist
----------
* We get ``r x[wrong[1L]]``
* We get ``r x[wrong[2L]]``
* We get ``r x[idx]``
Solution
========
`x[`r idx`]` returns the `r idx`th element of the vector `x`
which is `r x[idx]`.
Answerlist
----------
* False
* False
* Correct
Meta-information
================
exname: Vector subsetting
extype: schoice
exsolution: 001
exshuffle: True
Again, it is recommended to check the question multiple times by calling:
# Test html output and meta information (correct answer)
exams_metainfo(exams2html("vector-subsetting.Rmd"))
In contrast to the static single choice exercise this exercise uses a randomized vector x
, randomized vector indices (idx
), and randomized correct values (res
). Thus, in addition to the ‘runtime’, ‘position of correct solution’ and ‘rank of correct solution’ the function also returns the values of all scalars (here idx
, res
) of all randomizations during the stress test.
## Run the stresstest
#stress <- stresstest_exercise("vector-subsetting.Rmd")
## Plot overview
#plot(stress)
The additional information stored on stress$objects
can be used to get more insights, e.g., how often each index idx
\(\in {1, ..., 5}\) has been chosen and what the correct answers were (res
):
## Plot histograms of the two scalars idx/res from the
## 100 randomizations performed by the stresstest function
#par(mfrow = c(1, 2), mar = c(5, 5, 2, 1))
#hist(stress$objects$idx, col = "gray90",
# breaks = seq(0.5, 5.5, by = 1),
# main = "Histogram of idx")
#hist(stress$objects$res, col = "gray90",
# main = "Histogram of res")
Once the questions have been tested and approved we can create the final test used to upload to OpenOLAT. This is done by calling the exams2openolat()
function. The first input can be a single file (e.g., iso3-code.Rmd
; results in a test with only one question) or a list of files. Each entry of the list contains the name of one .Rmd
file and corresponds to one question in the quiz. If the list entry contains a character vector with multiple file names, R/exams will chose one of the files provided to generate the question (allows for additional randomization of questions).
Note: exams2openolat()
has a wide range of additional input arguments to adjust/control the test settings (see R/exams). To generate the zip archive file for OpenOLAT we have to call the following:
# Using the c403 package
library("c403")
# Set seed (makes the randomization reproduceable
set.seed(321)
# Generate OpenOLAT test/exam resource using the c403 package
files <- list("iso3-code.Rmd", "vector-subsetting.Rmd")
c403::exams2openolat(file = files, quiet = TRUE,
n = 3L, name = "test-quiz",
maxattempts = 100L)
file
: the files with the questions to be converted.n
: number of randomized tests to be generated. Default is n = 1L
. If you create n = 10L
tests for 50 participants each test might be re-used about 5 times (n
does not limit the number of participants).name
: name of the test (will generate <name>.zip
as output file; default is ).maxattempts
: number of attempts - number of times the participants are allowed to change the selected answer. Default is only 1 (maxattempts = 1L
).The function exams2openolat()
from the c403
package automatically creates two files: the zip file (test-quiz.zip
) to be uploaded to OpenOLAT and the file test-quiz.rds
which contains the randomized questions/tests.
Alternatively one can use the exams2openolat()
function from the exams
package. Note: exams::exams2openolat()
does not automatically save the randomized questions (.rds
file), however, it is very important to store this information if one wants to evaluate the tests later on. It is also highly recommended to set a seed for pseudo-randomization. This allows to reproduce the generated tests if the .rds
file gets lost. If one does not want to use the c403
package you can do it as follows using the exams
package:
# -------------------------------------
# Or: load the exams package
# -------------------------------------
suppressPackageStartupMessages(library("exams"))
# Set seed (makes the randomization reproduceable
set.seed(321)
# Generate OpenOLAT test/exam resource
exams <- exams:exams2openolat(file = files,
n = 3L, name = "test-quiz-exams",
maxattempts = 100L)
# Save the generated tests for evaluation
saveRDS(exams, file = "test-quiz-exams.rds")
Note that c403::exams2openolat()
and exams::exams2openolat()
have different default arguments (e.g., the c403
package uses the German language by default).
Once the test has been generated one can import the tests to OpenOlat. To do so, log in to OpenOLAT and go to the Authoring section. The zip file (test-exam.zip
) can be uploaded via the Import button:
It is highly recommended to give the learning resource a clear and unique title such that one can easily find the resource later on. Once the test has successfully been imported a new course element has to be created. First, navigate to your course in OpenOLAT and enter the Administration area. Afterwards there are two options:
The latter option has the advantage that it also copies the settings of the course element! To ‘attach’ the uploaded learning resource select the course element, click on Test configuration and select or replace (in case you made a copy of an existing test course element) the ‘file’ (the uploaded test/zip file).
Note: check and adjust all test settings from Title and description, accessibility, and course element options!