Academic Journals Database
Disseminating quality controlled scientific knowledge

Examinee-centered standard setting for large-scale assessments: The prototype group method

ADD TO MY LIST
 
Author(s): Thomas Eckes

Journal: Psychological Test and Assessment Modeling
ISSN 2190-0493

Volume: 54;
Issue: 3;
Start page: 257;
Date: 2012;
VIEW PDF   PDF DOWNLOAD PDF   Download PDF Original page

Keywords: standard setting | Rasch measurement | prototypes | large-scale assessment | language proficiency

ABSTRACT
This paper presents the prototype group method (PGM) of standard setting within the context of a large-scale language assessment project. The PGM combines a Rasch measurement approach to the analysis of examinee proficiency with the concept of prototypes drawn from research on human judgment and categorization. Experts first identify learners typical of each of five levels of language proficiency as specified by the Common European Framework of Reference for Languages (CEFR; Council of Europe, 2001). Based on the distributions of proficiency estimates for learner prototypes belonging to adjacent levels, cut scores are computed by means of a logistic regression procedure. These cut scores define the language proficiency level a particular examinee has achieved. Data from 39 independent samples of examinees (total N = 8,721) covering a range of German language proficiency levels are used to illustrate the PGM. Rasch analysis and logistic regression results corroborate the adequacy of this approach. The discussion focuses on the method’s distinctive features, practical requirements of its implementation, and issues of cut-score validation.
RPA Switzerland

Robotic Process Automation Switzerland

    

Tango Rapperswil
Tango Rapperswil