Exams are a central instrument for assessing the individual competencies of students. They provide a significant incentive to study. They ensure the quality of training programmes by bringing experts together to reflect on what students should actually be learning in line with the training objectives. This way, they exert a positive influence on the refinement of these objectives. Overall, the quality of exams mirrors the commitment of the educational institutions responsible for training the students.

In the field of medicine, two types of exams have emerged as particularly significant: the Objective Structured Clinical Examination (OSCE), which assesses clinical skills, and the written multiple-choice exam, which evaluates medical knowledge.

The OSCE is a practice-oriented form of exam designed to evaluate the clinical abilities students have acquired. In an OSCE, students are presented with an authentic clinical problem (e.g., a patient who arrives in the emergency department with a cough and fever). The patient’s concerns and symptoms are convincingly portrayed by actors in the role of standardised patients. An OSCE typically consists of eight to twelve successive clinical tasks, called "stations." In this way, students’ competencies can be independently assessed in different clinical situations by multiple examiners, which significantly enhances the reliability of the exam scores.

Development and Preparation: Designing an OSCE is a complex process involving numerous individuals from a range of disciplines. First, based on the training objectives for the relevant educational programme, domain experts determine which clinical skills need to be assessed. Building on this, various stations are devised to simulate specific scenarios drawn from medical practice. These include taking a patient’s history, performing physical exams, and deciding on the next diagnostic and therapeutic steps. Professional and patient-focused communication is a central aspect of every station. Developing detailed case descriptions and corresponding marking schemes tailored precisely to each patient presentation is crucial for ensuring that examiners can fairly and reliably evaluate students’ performance.

In the subsequent quality control phase, which includes multiple steps, the stations are reviewed through peer review. Quality control comprises joint workshops with OSCE methodology specialists and experienced clinicians, as well as a final evaluation of the correctness by an exam board. Once the optimised patient scenarios have been completed, the standardised patients practise intensively to portray symptoms and behaviours consistently, so that all students are assessed under comparable conditions.

Conducting the Exam: Proper logistics are vital for the smooth running of an OSCE exam. Rooms must be arranged, timetables devised, and a sufficient number of examiners recruited, invited, and briefed. Each station is allotted a precisely defined time slot, after which students rotate to the next station. Examiners must be both clinically proficient and trained to use the rating instruments properly so that students demonstrating the same level of performance receive the same score. Exam performance is recorded digitally according to predefined criteria.

Evaluation: After the exam, the digitally recorded rating forms are analysed. Statistical methods are employed to guarantee the reliability of the exam and the validity of its results. Any irregularities, such as ambiguous marking criteria or unusually high or low marks from individual examiners, are identified and taken into account for the benefit of the students. Additionally, feedback from both examiners and students is evaluated in order to refine each station and improve future iterations of the exam as a whole.

Multiple-choice exams are widespread in medicine due to their efficiency and objectivity. Well-designed multiple-choice questions presenting nuanced patient scenarios can measure cognitive competencies related to clinical decision-making as well as the application of medical knowledge.

Development and Preparation: Writing high-quality multiple-choice questions is an art in itself. A patient scenario must be described in such a way that a particular diagnosis is clearly the most likely, or that a specific medical approach is clearly the best option in the given circumstances, without it being obvious from the outset what action is required. The question that follows must be formulated unambiguously to avoid misunderstandings. The answer choices must include the correct solution and several options that may appear plausible but are definitively less appropriate. These so-called ‘distractors’ represent typical misconceptions about the patient’s situation and thus test students’ real understanding. Another challenge is to phrase the questions so that they provide no unintended clues about the correct answer.

Conducting the Exam: To safeguard the purpose of the exam—namely, to measure students’ competencies—and to maintain fairness, cheating and other dishonest methods must be prevented. Clear rules, instructions at the start of the exam, surrender of mobile devices, and a spacious seating plan can help achieve this. A quiet, orderly setting is also aimed for, so that students can focus fully on their tasks. Thanks to software developed by the IML, multiple-choice exams can be efficiently administered and evaluated.

Evaluation: Once thorough double-checks have been completed, exam results are communicated to students. Behind the scenes, detailed item analyses are performed, calculating statistical indices such as the difficulty index and item discrimination to gauge the quality of each question. Here, too, feedback from students and examiners is closely reviewed to identify ways to enhance the exam in future.

Both exam formats face the same challenge: how to ensure that they remain relevant and reliable in the long term. An exam that makes clinical sense and is methodologically sound not only assesses the relevant competencies of students more accurately but also provides students with a much better incentive to learn, helping them prepare optimally for a career in medicine. How, then, can exams be continually improved and kept up to date?

This must primarily be addressed on a content level by constantly asking which new learning objectives—driven by changes in society and medicine—should be included, and which content can be removed from the exam because it is outdated. It also requires regular training in exam methodology for those involved in creating exams and evaluating exam performance. The use of state-of-the-art hard- and software supports the efficiency of the exam process and the objectivity of the assessments. Ongoing improvements in rating criteria, informed by psychometric research, are also part of this continuous development.

The development, preparation, administration, and evaluation of OSCE and multiple-choice exams represent a complex endeavour requiring meticulous planning and multifaceted expertise. By applying recognised standards and continually improving both content and methodology, exams can be designed to meet the highest demands. In doing so, they contribute to training competent doctors who are well prepared for professional practice.

The IML has been active for over 50 years as a national centre of expertise in exam services, taking a leading role in more than 240 exams in Switzerland. It supports the University of Bern, the federal government, and many other clients by providing its expertise in designing, conducting, and analysing exams of the highest quality.

Publications

Schurter T, Escher M, Gachoud D, Bednarski P, Hug B, Kropf R, Meng-Hentschel J, König B, Beyeler C, Guttormsen S, Huwendiek S. Essential steps in the development, implementation, evaluation and quality assurance of the written part of the Swiss federal licensing examination for human medicine. GMS J Med Educ. 2022 Sep 15;39(4):Doc43. doi: 10.3205/zma001564.

Wagner FL, Feller S, Schmitz FM, Zimmermann PG, Krings R, Guttormsen S, Huwendiek S. Usability and preference of electronic vs. paper and pencil OSCE checklists by examiners and influence of checklist type on missed ratings in the Swiss Federal Licensing Exam. GMS J Med Educ. 2022 Apr 14;39(2):Doc24. doi: 10.3205/zma001545.

Lahner FM, Schauber S, Lörwald AC, Kropf R, Guttormsen S, Fischer MR, Huwendiek S. Measurement precision at the cut score in medical multiple choice exams: Theory matters. Perspect Med Educ. 2020 Aug;9(4):220-228. doi: 10.1007/s40037-020-00586-0.

Monti M, Klöckner-Cronauer C, Hautz SC, Schnabel KP, Breckwoldt J, Junod-Perron N, Feller S, Bonvin R, Huwendiek S. Improving the assessment of communication competencies in a national licensing OSCE: lessons learned from an experts' symposium. BMC Med Educ. 2020 May 26;20(1):171. doi: 10.1186/s12909-020-02079-4.

Guttormsen, S., Beyeler, C., Bonvin, R., Feller, S., Schirlo, C., Schnabel, K. P., Schurter, T., & Berendonk, C. (2013). The new licencing examination for human medicine: from concept to implementation. Swiss Med Wkly, 143(w13897), 1–10.doi:10.4414/smw.2013.13897.