In March 2017, around 50 subject experts from all five medical faculties in Switzerland came together to develop the new exam questions for the Federal licensing MC examination in human medicine. It was the first time that the workshop was conducted in this form. The concept was developed by the national working group (AG MC) and the Institute for Medical Education (IML) in order to optimize the question production. The methodology experts of the IML led the workshop on site.

New way of producing questions

In previous years, the exam questions were created partly by the experts individually, partly within internal workshops in the individual faculties, and partly in central half-day workshops. They were then checked and corrected in four central half-day revision sessions. All of these individual steps have now been merged into a 2-day workshop.

Prior to the current “retreat”, the participants were asked to develop ideas and drafts for new exam questions and to bring them along to the workshop. To this aim, they received written instructions on the required form and the desired content of the questions. On the first day of the workshop, after a joint introduction, these drafts were discussed and revised in small groups (3-4 persons). The mixing of the subject areas and the balanced composition of the groups occurred by drawing lots. Past experience has shown that subject-specific expert groups more frequently lead to subject-specific questions, which go significantly beyond the examinees’ level at the end of the medical degree.

On the second day, the participants were divided into new, somewhat larger groups (5-7 persons). This ensured that there was a representative of each faculty in every group, that the subject areas were again well mixed, and that each group contained at least one subject expert with long-standing familiarity with this exam and its requirements. In these revision groups, all questions developed on the previous day were again checked and revised. In addition, the groups checked whether all of the 5-6-year-old questions from the pool were still up to date.

Output markedly better than in previous years

Over the two days, around 250 new questions were created and revised as regards content, and 360 existing questions from the pool were checked. This quantitative annual output is considerably higher than in the old system. In addition, the overall quality of the new questions was considerably improved.

What are the reasons for this success?

• The participants knew in advance that the question drafts they brought along would be discussed in the group on the first day.
• For two days, the participants were completely removed from their everyday work. This enabled them to work in a concentrated manner without distraction.
• The mixing of the subject areas and the faculties enabled questions to be formulated at an appropriate level from the outset.
• The representatives of each faculty in the national working group for this exam showed tremendous commitment in recruiting the participants from their faculties.
• The workshop was supported by the national examination board and the clinic directors, making the experts’ participation possible in the first place.
• The welcome by the President of the national examination board demonstrated to the participants that their work was important and highly appreciated by the board.
• It was possible to record the questions in the IML pool on site, allocate them to the revision groups, revise them, and comment on them.

Due to the positive feedback (see participants’ evaluation/right-hand box), the workshop is to be carried out again using the same format in the coming year.

We thank the Federal Office of Public Health and the examination board for their great support of this project, and the members of the working group MC as well as the participating question authors for their important contribution to its success.