Academic Journals Database
Disseminating quality controlled scientific knowledge

ACCOUNTING FOR USERS' INFLATED ASSESSMENTS OF ONLINE CATALOG SEARCH PERFORMANCE AND USEFULNESS: AN EXPERIMENTAL STUDY

ADD TO MY LIST
 
Author(s): Charles R. Hildreth

Journal: Information Research: an international electronic journal
ISSN 1368-1613

Volume: 6;
Issue: 2;
Start page: 101;
Date: 2001;
Original page

Keywords: user-orientation | users | information retrieval | IR | performance | evaluation | user satisfaction | search results | utility | on-line library catalogues | catalogs | OPACs | false positives | interface | ease of use | system | usefulness | satisfaction | Web | search task | difficulty | personal characteristics

ABSTRACT
User-oriented approaches to information retrieval (IR) system performance evaluation assign a major role to user satisfaction with search results and overall system performance. Expressed satisfaction with search results is often used as a measure of utility. Many research studies indicate that users of on-line library catalogs (OPACs) and other IR systems often express satisfaction with poor search results. This phenomenon of "false positives," inflated assessments of search results and system performance, has not been adequately explained. Non-performance factors such as interface style and ease of use may have an affect on a searcher's satisfaction with search results. The research described in this report investigates this phenomenon. This paper presents the findings of an experimental study which compared users' search performance and assessments of ease of use, system usefulness, and satisfaction with search results after use of a Web OPAC or its conventional counterpart. The primary questions addressed by this research center on the influence of two experimental factors, OPAC search interface style and search task level of difficulty, on the dependent variables: actual search performance, perceptions of ease of use and system usefulness, and assessments of satisfaction with search results. The study also investigated associations between perceived ease of use, system usefulness, and satisfaction with search results. Lastly, the study looked for associations between the dependent variables and personal characteristics. No association was found between satisfaction with search results and actual search performance. Web OPAC searchers outperformed Text OPAC searchers, but search task level of difficulty is a major determinant of search success. A strong positive correlation was found between perceptions of system ease of use and assessments of search results.
Why do you need a reservation system?      Save time & money - Smart Internet Solutions