Generalizability theory analysis of reliability of scores assigned to students in essay mathematics examinations with marking scheme in Akwa Ibom State

Abstract

This study examined reliability of scores assigned to students in essay mathematics examinations with marking scheme based on generalizability theory in Akwa Ibom State. Two research questions were raised and a two-facet crossed design of the generalizability theory is used. The sample size for this study comprised 100 Senior Secondary Ill (SS Il) students and two mathematics teachers (raters) Mho were randomly sampled in Abak Metropolis of Akwa Ibom State, Nigeria. Mathematics Essay Promotion Examination developed and administered by the Akwa Ibom State Ministry of Education for SS Il students with inter-rater reliability index was 0.72 was adapted and used as instrument for data collection. The collected data was analyzed using general linear model and the results obtained revealed that the generalizability and decision study variance components and interactions varied among students, items and raters based on their contribution to the analysis. The generalizability and dependability coefficient were found to be 0.69 and 0.69 respectively indicated high reliabilities. It was concluded that in adopting generalizability theory analysis, desirable reliability estimates could be achieved for scoring essay examinations. It was recommended that instead of using a single major essay examination during a term, schools should divide it into several smaller examinations and make use of multiple raters to grade each student response as these approaches have the advantage of improving reliability of scores.