EBA item fit test

IRT models describe the situation where a vector \(\boldsymbol{x}=(x_{i})_{i=1,\ldots,k}\) of dichotomous item responses, taking values 0 and 1, depend on an underlying latent variable \(\theta\) (Van der linden 1997). A uni-dimensional parametric IRT model is based on four assumptions: (i) \(\theta\in(-\infty,\infty)\), (ii) local independence

\begin{equation} \label{LI}p(\boldsymbol{x}|\theta)=\prod_{i=1}^{k}p(x_{i}|\theta),\nonumber \\ \end{equation}

(iii) the absence of Differential Item Functioning (DIF; (Holland 1993))

\begin{equation} \label{noDIF}p(\boldsymbol{x}|\theta,y)=p(\boldsymbol{x}|\theta)\nonumber \\ \end{equation}

for any exogenous variable \(y\), and (iv) monotonicity of the functions \(\theta\mapsto p(x_{i}=1|\theta)\). Testing these assumptions is crucial before the items are used for measurement. Overall tests and tests aimed specifically at individual measurement assumptions are needed for this.

The Rasch model (Rasch 1960) is the simplest IRT model. It can be written

\begin{equation} \label{RM}p(\boldsymbol{x}|\theta)=K(\boldsymbol{\beta},\theta)\exp(r(\boldsymbol{x})\theta-\boldsymbol{x}^{T}\boldsymbol{\beta})\\ \end{equation}

where \(\boldsymbol{\beta}=(\beta_{i})_{i=1,\ldots,k}\) is a vector of item parameters and \(r(\boldsymbol{x})=\sum_{i=1}^{k}x_{i}\) is the sum score, seen from (\ref{RM}) to be a sufficient statistic for \(\theta\). The conditional probabilities

\begin{equation} p(\boldsymbol{x}|R=r)=\frac{\exp(-\boldsymbol{x}^{T}\boldsymbol{\beta})}{\gamma(r,\boldsymbol{\beta})}\nonumber \\ \end{equation}

where

\begin{equation} \gamma(r,\boldsymbol{\beta})=\sum_{\boldsymbol{x}:r(\boldsymbol{x})=r}\exp(-\boldsymbol{x}^{T}\boldsymbol{\beta}).\nonumber \\ \end{equation}

Maximizing the conditional log likelihood

\begin{equation} l(\boldsymbol{\beta})=-\sum_{v=1}^{N}\boldsymbol{x}_{v}^{T}\boldsymbol{\beta}-\sum_{v=1}^{N}\log\gamma(r_{v},\boldsymbol{\beta})\nonumber \\ \end{equation}

yields item parameter estimates that are conditionally consistent (Andersen 1970)

A survey of item fit statistics

This section provides an overview of existing item fit statistics. Let \(\boldsymbol{p}_{v}=E(\boldsymbol{X}|\theta=\hat{\theta}_{v})\) and \(\boldsymbol{p^{(c)}}_{v}=E(\boldsymbol{X}|R=r_{v})\) denote the expected value of the vector of item responses conditional on the estimated value of \(\theta\) and on the observed value of the total score, respectively.

Let \(D_{v}=\textrm{diag}(\hat{\boldsymbol{p}}_{v})\) and \(D_{v}=\textrm{diag}(\boldsymbol{p}^{(c)}_{v})\) and let, for \(i=1,\ldots,k\), \(p_{iv}\) and \(p^{(c)}_{iv}\) denote the elements of the vectors \(\boldsymbol{p}_{v}\) and \(\boldsymbol{p^{(c)}}_{v}\), respectively.