Pass-Fail Decisions for Borderline Performers After a Summative Objective Structured Clinical Examination
To determine what expert assessors value when making pass-fail decisions regarding pharmacy students based on summative data from objective structured clinical examinations (OSCE), and to determine the reliability of these judgments between multiple assessors. All assessment data from 10 exit-from-d...
Saved in:
Published in: | American journal of pharmaceutical education Vol. 83; no. 2; pp. 6849 - 147 |
---|---|
Main Authors: | , , , |
Format: | Journal Article |
Language: | English |
Published: |
United States
Elsevier Inc
01-03-2019
American Association of Colleges of Pharmacy Elsevier Limited American Journal of Pharmaceutical Education |
Subjects: | |
Online Access: | Get full text |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Summary: | To determine what expert assessors value when making pass-fail decisions regarding pharmacy students based on summative data from objective structured clinical examinations (OSCE), and to determine the reliability of these judgments between multiple assessors.
All assessment data from 10 exit-from-degree OSCE stations for seven borderline pharmacy students (determined by standard setting methods) and one control was given to three of eight assessors for review. Assessors determined an overall pass-fail decision based on their perception of graduate competency. Assessors were interviewed to determine their decision-making rationale. Intraclass correlation coefficients were used to calculate reliability between assessor judgments.
Expert consensus was achieved for three of the eight students, however, the assessors’ decisions did not align with standard-setting results. The reliability of assessors’ decisions was poor. Assessors focused on ability to make correct recommendations rather than on gathering information or providing follow-up advice. Global evaluations (including a student’s communication skills) rarely influenced the assessors’ decision-making.
When faced with making pass-fail decisions for borderline students, the assessors focus on evaluating the same competencies in the students but differed in their expected performance levels of these competencies. Pass-fail decisions are primarily based on task-focused components instead of global components (eg, communication skills), despite that global components are weighted the same for scoring purposes. |
---|---|
Bibliography: | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 Note: At the time of manuscript submission, Dr. Wilby was affiliated with the College of Pharmacy, Qatar University. |
ISSN: | 0002-9459 1553-6467 |
DOI: | 10.5688/ajpe6849 |