Academic Journals Database
Disseminating quality controlled scientific knowledge

On the importance of using balanced booklet designs in PISA

Author(s): Andreas Frey | Raphael Bernhardt

Journal: Psychological Test and Assessment Modeling
ISSN 2190-0493

Volume: 54;
Issue: 4;
Start page: 397;
Date: 2012;
VIEW PDF   PDF DOWNLOAD PDF   Download PDF Original page

Keywords: booklet design | testing | large-scale assessment | item response theory | Programme for International Student Assessment

The effect of using a balanced compared to an unbalanced booklet design on major PISA results was examined. The responses of 39,573 students who participated in the PISA-E 2006 assessment in Germany were re-analyzed. Using an unbalanced booklet design instead of the original booklet design led to an increase in mean reading performance of about six points on the PISA scale and altered the gender gap in reading to different degrees in the 16 federal states of Germany. For students with an immigration background, the reading performance was significantly higher for the unbalanced design than for the original design. For the unbalanced design, the relationship between self-reported effort while taking the test and reading performance was higher compared to the original design. The results underline the importance of using a balanced booklet design in PISA in order to avoid or minimize bias in population parameters estimates.
RPA Switzerland

RPA Switzerland

Robotic process automation


Tango Rapperswil
Tango Rapperswil