Academic Journals Database
Disseminating quality controlled scientific knowledge

Standard setting in complex performance assessments: An approach aligned with cognitive diagnostic models

ADD TO MY LIST
 
Author(s): Robert W. Lissitz | Feifei Li

Journal: Psychological Test and Assessment Modeling
ISSN 2190-0493

Volume: 53;
Issue: 4;
Start page: 461;
Date: 2011;
VIEW PDF   PDF DOWNLOAD PDF   Download PDF Original page

Keywords: Standard setting | cognitive diagnostic models | analytical approach | evidence centered design | performance centered | multidimensional standards

ABSTRACT
With the increased interest in student-level diagnostic information from multiple performance assessments, it becomes possible to create multivariate classifications of knowledge, skills and abilities (KSAs). In this paper, a systematic, multivariate and non-compensating standard setting approach, called the cognitive analytical approach (CAA), is proposed for performance assessment with complex tasks. CAA is based on the framework of evidence-centered design (Mislevy, Steinberg, & Almond, 2003) that supports a chain of reasoning from design and development to delivery of an assessment. In CAA, the performance standards are established simultaneously with domain-modeling, test specifications, and item writing rather than after the assessment has been completed; the cut scores are evaluated iteratively along with the test design and development phases. CAA has the benefits of ensuring the validity of the performance standards, reducing the cognitive load of standard setting, including the complexity of the tasks, and facilitating the vertical articulation of KSAs. In this paper, we elucidate the theoretical and practical rationale of CAA and demonstrate its proce-dures and results with an illustrative example.
Why do you need a reservation system?      Affiliate Program