Comparative study of classical test theory and item response theory using item analysis results of quantitative chemistry achievement test

Abstract

Classical Test Theory (CTT) and Item Response Theory (IRT) are the two major test theories used to analyse test responses. This study compared the results from the CTT and IRT 3 parameter Logistic model analyses to ascertain which is better fit for the analysis of Quantitative Chemistry Achievement Test (QCAT) items. Fifty objective test items on Senior Secondary 2 quantitative chemistry (calculations in chemistry) were used and Survey research-type of non-experimental design was adopted. Randomly selected sample of 1105 students participated. Item analyses were done based on CTT and IRT using Bilog MG, Dimtest and R software. The QCAT items certified the IRT assumptions on dimensionality (unidimensional), local independence and model-data fit (3

parameter logistic model). Items from CTT analysis with rpbs≥0.20 and 0.30≤p≤0.80 were considered good items and selected. Items from IRT analysis with -3≤b ≤+3 were considered good items and selected. The result revealed that only 11 items survived the CTT analysis while 20 items

survived the IRT analysis. The reliability of the items from CTT (0.27) was lower than that of IRT (0.60). IRT is a more effective method of item analysis for objective test item responses and should be used by scale developers.