Handbook for setting up courses

UNION EUROPÉENNE DES MÉDEC INS SPÉC IAL I STES EUROPEAN UNION OF MEDI CAL SPEC IAL I STS Association internationale sans but lucratif – International non-profit organisation

1) The “ absolute ” pass mark (expressed as a number or percentage of questions needed to be answered correctly, e.g. 70%) a. This is criterion reference based and therefore it is possible for all candidates to pass or fail 2) The “relative ” pass mark (expressed in relation to a number or percentage of individuals taking the exam, e.g. top 33%) a. This is norm-reference based therefore the grade of the examinee is determined by both his/her own achievement compared against the achievement of all other candidates. 3) The “ Angoff ” pass mark. The Angoff score is determined after the question has been written, reviewed and accepted as appropriate for the examination. a. It is determined before the examination by a set of experts (who may or may not be the part of the question writing group). b. It is the likelihood of borderline competent candidate whose knowledge, skills and abilities are considered just sufficient to pass the examination overall of correctly answering that individual question. c. A good examination will consist of a series of questions with a wide distribution of Angoff scores. d. “ Modified Angoff ” scores are determined in a similar way but are determined after the examination has been sat by the test takers and the performance of each question is known and taken into account by those determining the modified Angoff score. e. A combination of Angoff and modified Angoff scores can be used to determine the passmark for a particular diet of an examination. f. Questions where the Angoff or score is significantly at odds with the actual performance of the question in the exam should be reviewed to determine if the question is faulty or requires further revision.

Finally:

Whilst the generation of new questions is essentially a task for the individual, the process of maximising the quality of MCQs is a task for a group of motivated and experienced MCQs writers who will, at times, need to meet face-to-face.

Experience suggests that the face-to-face format, although more costly, facilitates an international atmosphere and understanding. It encourages individual contributors and is considered irreplaceable, not only to motivate individual writers but to maximise the intellectual benefit from each newly developed MCQ. Discussion of individual topics should be encouraged and the contribution of all individuals valued. In particular those who are inexperienced questions writers should be facilitated in a constructive manner to enhance their individual contribution to the joint

A VENUE DE LA C OURONNE , 20 14

T +32 2 649 51 64 F +32 2 640 37 30 info@uems.net

BE- 1050 BRUSSELS

www.uems.net