SMALL SAMPLE ESTIMATION OF ITEM PARAMETERS IN ITEM RESPONSE THEORY MODELS USING OPERATIONAL DATA

Abstract

Item Response Theory (IRT) models have been widely used to analyse test data and develop IRT-based tests. An important requirement in applying IRT models is the stability and accuracy of model parameters. Substantial research work has been undertaken in the past to study the effect of sample size on the estimation of IRT model parameters using simulations. One of the limitations of using pure simulations to study the effect of sample size on IRT item parameter estimation is that the model assumptions are strictly met, which is seldom true for operational test data. However, data from operational tests do not normally strictly meet the model assumptions. It was therefore in the interest of this study to use real data in comparing item parameter estimates from different samples sizes so that the possible minimum sample size could be determined for application in IRT dichotomous models. The study compared three sample sizes of: 250,500 and 1000 obtained by administering a 60 item multiple choice test to 1750 MSCE students across Zomba City. The analysis was done using ANOVA in SPSS. At 95% confidence interval the results showed that the item parameter estimates obtained from the three independent samples were statistically the same. This lead to the conclusion that a sample of size 250 can be employed in IRT’s 2PLM and 1PLM to obtain item parameter estimates that are statistically equivalent to those from larger samples.