Start to write R/exams questions
R/exams questions are written in either pure markdown (.md), R/markdown (.Rmd) or R/LaTeX using Sweave (.Rmd). For demonstration purposes this tutorial is only using R/Markdown files, more information can be found in the [First Steps][rexams-firsteps] tutorial on r-exams.org.
Write your first static question
To create a new question we write a new .Rmd file and edit the content with a text editor, e.g., a file called iso3-code.Rmd. The following output shows the content of this file (iso3-code.Rmd) which contains a simple ‘static’ single choice R/exams question:
Question
========
What is the ISO3 country code for Austria.
Answerlist
----------
* AUT
* AUS
* USA
Solution
========
The ISO3 country code for Austria is **AUT**.
Answerlist
----------
* Correct answer.
* False, AUS stands for Australia.
* False, USA stands for the United States of America.
Meta-information
================
exname: ISO3 Country Code
extype: schoice
exsolution: 100
exshuffle: TrueEach question consists of three sections, namely “Question”, “Solution”, and “Meta-information”.
- Question: contains the question itself, simply text which contains the task. The “Answerlist” subsection defines the possible answers (here 3).
- Solution: this is typically not shown to the participants, or only depending on the system settings (e.g., if the ‘solutionswitch’ in OpenOLAT is enabled). Can contain text with an extensive solution if needed, and contains a subsection “Answerlist” in the same order as the possible answers in the “Question” section.
- Meta-information: contains the name and type of the question, information about the correct solution, and additional settings. In this case it is a single choice question (
extype: schoice),exsolutiondefines the correct answer. Three possible answers, the first one is correct (thus100).exshuffle: Truetells R/Exams to shuffle the possible questions (3 out of 3; random order).
Testing the question
Once we have saved our file (iso3-code.Rmd) we can test the question by calling:
# Renders the question as html
exams2html("iso3-code.Rmd")This will open the html-rendered version of the question in your browser. As exshuffle is turned on you will get a different randomization every time the command above is called. It is suggested to do this multiple times to see if the randomization works as expected.
Testing the solution
The function exams_metainfo() allows to check/test if the solution defined in the Meta-information section is specified correctly.
# Test solutionstring
exams_metainfo(exams2html("iso3-code.Rmd"))## Loading required namespace: base64enc##
## exam1
## 1. ISO3 Country Code: bThe output shows the correct answer is “2” which can be compared with the rendered html output (not shown here) to see if it is correct.
Stress-testing the question
Once one has checked that the randomization works, the texts are correct, and the answer is correct a stress-test for the question can be done by calling stresstest_exercise().
stresstest_exercise() renders the exercise multiple types (100 times by default) and returns some information about the randomization.
# Stresstesting 'iso3-code.Rmd' (100 times)
stress <- stresstest_exercise("iso3-code.Rmd")
plot(stress)
The output shows the runtime to generate the randomized questions and the position of the correct solution. As we use exshuffle: True with three possible answers this should be close to a uniform distribution between 1, 2, and 3.
Write our first dynamic question
R/exams questions can also be dynamic. The following example is the content of a second question stored in vector-subsetting.Rmd.
```{r generate question}
# Create a random integer vector of length 5
x <- sample(1:100, 5)
# Ask for the following index between 1:5
idx <- sample(1:5, 1)
# Wrong answers
wrong <- sample((1:5)[-idx], 2)
# The correct answer:
res <- x[idx]
```
Question
========
Vector subsetting in _R_. We can access elements/values
in a vectur using vector subsetting by index. Imagine we
have the following vector `x`:
```{r}
print(x)
```
If we call `x[`r idx`]`, what is the resulting value?
Answerlist
----------
* We get ``r x[wrong[1L]]``
* We get ``r x[wrong[2L]]``
* We get ``r x[idx]``
Solution
========
`x[`r idx`]` returns the `r idx`th element of the vector `x`
which is `r x[idx]`.
Answerlist
----------
* False
* False
* Correct
Meta-information
================
exname: Vector subsetting
extype: schoice
exsolution: 001
exshuffle: TrueTesting the question/solution
Again, it is recommended to check the question multiple times by calling:
# Test html output and meta information (correct answer)
exams_metainfo(exams2html("vector-subsetting.Rmd"))Stress-testing
In contrast to the static single choice exercise this exercise uses a randomized vector x, randomized vector indices (idx), and randomized correct values (res). Thus, in addition to the ‘runtime’, ‘position of correct solution’ and ‘rank of correct solution’ the function also returns the values of all scalars (here idx, res) of all randomizations during the stress test.
## Run the stresstest
#stress <- stresstest_exercise("vector-subsetting.Rmd")
## Plot overview
#plot(stress)The additional information stored on stress$objects can be used to get more insights, e.g., how often each index idx \(\in {1, ..., 5}\) has been chosen and what the correct answers were (res):
## Plot histograms of the two scalars idx/res from the
## 100 randomizations performed by the stresstest function
#par(mfrow = c(1, 2), mar = c(5, 5, 2, 1))
#hist(stress$objects$idx, col = "gray90",
# breaks = seq(0.5, 5.5, by = 1),
# main = "Histogram of idx")
#hist(stress$objects$res, col = "gray90",
# main = "Histogram of res")
