Item analysis of a reading test in a Sri Lankan context using classical test theory

Abstract

This paper is based on a research study on a reading test that evaluates the different cognitive processes prescribed by Khalifa and Weir (2009). The 25-item test was designed based on a test specification targeted at the B2 level of the Common European Framework of Reference for Languages (CEFR). The responses of 50 students were used to check the validity and reliability of the test. The validity of the test was ascertained through item analysis involving item difficulty indices, item discrimination indices, and distractor analysis. Each item was studied to provide detailed information leading to the improvement of test construction. To achieve test reliability, the Kuder-Richardson Formula 20 (KR-20) was applied. The results were achieved by simply using Microsoft Excel. Findings revealed that the test met the standards for content validity, indicating acceptable item difficulty indices, with 17 items at the moderate level between the ranges of 0.30 and 0.79. Except for three items, all others functioned well to differentiate between high- and low-ability students, and only five items had malfunction distractors. Meanwhile, the reliability value of the test scores was 0.82, which is deemed a good value, proving the consistency of the test results. It signifies that more than half, that is 88%, of the test items were well functioning and that the test proved to be valid and reliable. The present research can contribute to students, teachers, and test-makers having an insightful understanding of item analysis and test development.