Handbook for setting up courses

UNION EUROPÉENNE DES MÉDEC INS SPÉC IAL I STES EUROPEAN UNION OF MEDI CAL SPEC IAL I STS Association internationale sans but lucratif – International non-profit organisation

Setting up a successful high-quality Exam

Since the hope of the UEMS is that the rigorous quality and standards imposed on the overall organization of these exams may persuade European National governments to recognize and accept these exams, it must be borne in mind that the basic structure of the exam process in question must be transparent to National or International regulators who are contemplating possible local recognition of the exam. It is apparent that writing good MCQs is both a science and an art. Item writing is an arduous task which requires not only mastery of the subject matter but also command of verbal communication skills and an understanding of the examination population.

The necessary requirements for a well-structured examination process based on MCQs include the following sections:

1) Developing new items on a continual basis (as assigned based on the needs of the examination and previously contributed material).

a) Professional item writers plan on one hour or more to write one good question.

b) When starting to write questions it is highly likely that the first few questions will require substantial modification by more experienced question writers in order to become good questions.

2) Reviewing and selecting items for inclusion in the written examination.

a) A review group (small in number) should be set up with the specific task of selecting questions which will cover, in number and content, the specific discipline of the exam.

b) The specific objective of the group is to select from the MCQ bank a range of questions appropriately distributed throughout the curriculum, with a range of perceived difficulty, and where available the use of “marker questions” known to perform well.

c) The use of "old" questions that have been validated provides a longitudinal standard and further validates the exam process.

d) The goal of the group is to draw upon a pool of examination items which are appropriate to measure the knowledge and skills necessary for safe and effective performance in the field of practice.

e) The recommended number of MCQs in an exam should not be less than 100 (for any specific specialty discipline) in order to achieve sufficient reliability and validity of the exam.

A VENUE DE LA C OURONNE , 20 11

T +32 2 649 51 64 F +32 2 640 37 30 info@uems.net

BE- 1050 BRUSSELS

www.uems.net