Item Response Theory Modeling and Scoring for Judgment Data

Psychometric analysis and scoring of judgment data using polytomous Item-Response Theory (IRT) models, as described in Myszkowski and Storme (2018) . A convenience function is used to automatically compare and select models, as well as to present a variety of model-based statistics. Plotting functions are used to present category curves, as well as information, reliability and standard error functions.


The goal of jrt is to provide tools to use Item-Response Theory (IRT) models on judgment data, especially in the context of the Consensual Assessment Technique. It is based on "mirt" (Chalmers, 2012) as its estimation engine.

Installation

You can install the released version of jrt from CRAN with:

install.packages("jrt")

Example use

  • Load the library
library(jrt)
  • To automatically select models
fit <- jrt(data, progress.bar = F)
#> The possible responses detected are: 1-2-3-4-5
#> 
#> Comparing models...
#> 
#> -== Automatic Model Selection ==-
#> AICc for Rating Scale Model: 4414.924
#> AICc for Generalized Rating Scale Model: 4310.307
#> AICc for Graded Response Model: 4007.604
#> AICc for Partial Credit Model: 4027.701
#> AICc for Generalized Partial Credit Model: 4021.567
#>  -> The best fitting model is the Graded Response Model.
#> 
#>  -== General Summary ==-
#> - 6 Judges
#> - 300 Products
#> - 5 response categories (1-2-3-4-5)
#> - Mean judgment = 2.977 | SD = 0.862
#> 
#> -== IRT Summary ==-
#> - Model: Graded Response Model (GRM; Samejima, 1969) | doi: 10.1007/BF03372160
#> - Estimation package: mirt (Chalmers, 2012) | doi: 10.18637/jss.v048.i06
#> - Estimation algorithm: Expectation-Maximization (EM; Bock & Atkin, 1981) | doi: 10.1007/BF02293801
#> - Method of factor scoring: Expected A Posteriori (EAP)
#> - AIC = 4000.69 | AICc = 4007.604 | BIC = 4111.803 | SABIC = 4000.69
#> 
#> -== Model-based reliability ==-
#> - Empirical reliability | Average in the sample: .896
#> - Expected reliability | Assumes a Normal(0,1) prior density: .896

  • To select models a priori
fit <- jrt(data, irt.model = "PCM")
#> The possible responses detected are: 1-2-3-4-5
#> 
#>  -== General Summary ==-
#> - 6 Judges
#> - 300 Products
#> - 5 response categories (1-2-3-4-5)
#> - Mean judgment = 2.977 | SD = 0.862
#> 
#> -== IRT Summary ==-
#> - Model: Partial Credit Model (PCM; Masters, 1982) | doi: 10.1007/BF02296272
#> - Estimation package: mirt (Chalmers, 2012) | doi: 10.18637/jss.v048.i06
#> - Estimation algorithm: Expectation-Maximization (EM; Bock & Atkin, 1981) | doi: 10.1007/BF02293801
#> - Method of factor scoring: Expected A Posteriori (EAP)
#> - AIC = 4022.957 | AICc = 4027.701 | BIC = 4115.551 | SABIC = 4022.957
#> 
#> -== Model-based reliability ==-
#> - Empirical reliability | Average in the sample: .889
#> - Expected reliability | Assumes a Normal(0,1) prior density: .759

  • To plot all category curves
jcc.plot(fit)
  • To plot on judge's category curves
jcc.plot(fit, judge = 1)
  • Graphical options
jcc.plot(fit, judge = 1, overlay.reliability = T, greyscale = T, theme = "classic")
  • To plot total information
info.plot(fit)
  • To plot judge information
info.plot(fit, judge = 1)
  • Other options for information plots
info.plot(fit, type = "Reliability",
          max.y = 1,
          y.line = .70,
          theta.span = 4,
          theme = "classic")

News

jrt 1.0.0

  • Initial release with functions jrt(), jcc.plot() and info.plot()

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("jrt")

1.0.0 by Nils Myszkowski, 4 months ago


Browse source code at https://github.com/cran/jrt


Authors: Nils Myszkowski [aut, cre]


Documentation:   PDF Manual  


GPL-3 license


Imports mirt, psych, irr, methods, utils, stats, dplyr, tidyr, ggplot2, ggsci

Depends on directlabels

Suggests knitr, rmarkdown


See at CRAN