Pass-Fail Decisions for Borderline Performers After a Summative Objective Structured Clinical Examination

To determine what expert assessors value when making pass-fail decisions regarding pharmacy students based on summative data from objective structured clinical examinations (OSCE), and to determine the reliability of these judgments between multiple assessors. All assessment data from 10 exit-from-d...

Full description

Saved in:
Bibliographic Details
Published in:American journal of pharmaceutical education Vol. 83; no. 2; pp. 6849 - 147
Main Authors: Ali, Mayar, Pawluk, Shane A., Rainkie, Daniel C., Wilby, Kyle John
Format: Journal Article
Language:English
Published: United States Elsevier Inc 01-03-2019
American Association of Colleges of Pharmacy
Elsevier Limited
American Journal of Pharmaceutical Education
Subjects:
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:To determine what expert assessors value when making pass-fail decisions regarding pharmacy students based on summative data from objective structured clinical examinations (OSCE), and to determine the reliability of these judgments between multiple assessors. All assessment data from 10 exit-from-degree OSCE stations for seven borderline pharmacy students (determined by standard setting methods) and one control was given to three of eight assessors for review. Assessors determined an overall pass-fail decision based on their perception of graduate competency. Assessors were interviewed to determine their decision-making rationale. Intraclass correlation coefficients were used to calculate reliability between assessor judgments. Expert consensus was achieved for three of the eight students, however, the assessors’ decisions did not align with standard-setting results. The reliability of assessors’ decisions was poor. Assessors focused on ability to make correct recommendations rather than on gathering information or providing follow-up advice. Global evaluations (including a student’s communication skills) rarely influenced the assessors’ decision-making. When faced with making pass-fail decisions for borderline students, the assessors focus on evaluating the same competencies in the students but differed in their expected performance levels of these competencies. Pass-fail decisions are primarily based on task-focused components instead of global components (eg, communication skills), despite that global components are weighted the same for scoring purposes.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
Note: At the time of manuscript submission, Dr. Wilby was affiliated with the College of Pharmacy, Qatar University.
ISSN:0002-9459
1553-6467
DOI:10.5688/ajpe6849