Continuous Norming

Conventional methods for producing standard scores in psychometrics or biometrics are often plagued with "jumps" or "gaps" (i.e., discontinuities) in norm tables and low confidence for assessing extreme scores. The continuous norming method introduced by A. Lenhard et al. (2016), , generates continuous test norm scores on the basis of the raw data from standardization samples, without requiring assumptions about the distribution of the raw data: Norm scores are directly established from raw data by modeling the latter ones as a function of both percentile scores and an explanatory variable (e.g., age). The method minimizes bias arising from sampling and measurement error, while handling marked deviations from normality, addressing bottom or ceiling effects and capturing almost all of the variance in the original norm data sample.


The package cNorm provides methods for generating non-parametric regression based continuous standard scores, as f. e. for psychometric test development, biometrics (e. g. biological and physiological growth curves), and screenings in the medical domain. It is based on the approach suggested by A. Lenhard et al. (2016). For an in-depth tutorial please consult the project homepage https://www.psychometrica.de/cNorm_en.html

Approach

Conventional methods for producing test norms are often plagued with "jumps" or "gaps" (i.e., discontinuities) in norm tables and low confidence for assessing extreme scores. cNORM addresses these problems and also has the added advantage of not requiring assumptions about the distribution of the raw data: The standard scores are established from raw data by modeling the latter ones as a function of both percentile scores and an explanatory variable (e.g., age) through Taylor polynomials. The method minimizes bias arising from sampling and measurement error, while handling marked deviations from normality – such as are commonplace in clinical samples. Contrary to parametric approaches, it does not rely on distribution assumptions of the initial norm data and is thus a very robust approach in generating norm tables.

Installation

cNORM can be installed via

install.packages("cNORM", dependencies = TRUE)

Additionally, you can download a precompiled version or access the github development version via

install.packages("devtools")
devtools::install_github("WLenhard/cNORM")
library(cNORM)

Please report errors. Suggestions for improvement are always welcome!

Example

Conducting the analysis consists of four steps:

  1. Data preparation
  2. Establishing the regression model and selecting the parameters
  3. Validating the model
  4. Generating norm tables and plotting the results

cNORM offers function for all of these steps, helps in selecting the best fitting models and in generating the norm tables.

## Basic example code for modeling the sample dataset
library(cNORM)
 
# Start the graphical user interface (needs shiny installed)
# The GUI includes the most important functions. For specific cases,
# please use cNORM on the console.
cNORM.GUI()
 
# If you prefer the console, you can use the syntax as well
# Rank data within group and compute powers and interactions for the internal dataset 'elfe'
data.elfe <- prepareData(elfe)
 
# Find best fitting regression model
model.elfe <- bestModel(data.elfe)
 
# Plot R2 of different model solution in dependence of the number of predictors
plotSubset(model.elfe, type=0)        # plot R2
plotSubset(model.elfe, type=3)        # plot MSE
 
#  Visual inspection of the percentile curves of the fitted model
plotPercentiles(data.elfe, model.elfe)
 
# In order to check, how other models perform, plot series of percentile plots with ascending
# number of predictors up to 14 predictors.
plotPercentileSeries(data.elfe, model.elfe, end=14)
 
# Cross validation of number of terms with 20% of the data for validation and 80% training.
# Due to the time intensity, max terms is restricted to 10; 3 repetitions
cnorm.cv(data.elfe, max=7, repetitions=10)
 
# Print norm table (for grade 3, 3.2, 3.4, 3.6) with T scores from T = 25 to T = 75
normTable(c(3, 3.2, 3.4, 3.6), model.elfe, minNorm = 25, maxNorm = 75, step = 1)
 
# The other way round: Print raw table (for grade 3)
rawTable(3, model.elfe)
 
# start vignette for a complete walk through
vignette("cNORM-Demo", package = "cNORM")

cNORM offers functions to choose the optimal model, both from a visual inspection of the percentiles, as well as by information criteria and model tests:

In this example, a Taylor polynomial with power k = 4 was computed in order to model a sample of the ELFE 1-6 reading comprehension test (sentence completion task; W. Lenhard & Schneider, 2006). In the plot, you can see the share of variance explained by the different models (with progressing number of predictors). Adjusted R2, Mallow's Cp (an AIC like measure) and BIC is used (BIC is available through the option type = 2). The predefined adjusted R2 value of .99 is already reached with the third model and afterwards we only get minor improvements in adjusted R2. On the other hand, Cp rapidly declines afterwards, so model 3 seems to be a good candidate in terms of the relative information content per predictor and the captured information (adjusted R2). It is advisable to choose a model at the "elbow" in order to avoid over-fitting, but the solution should be tested for violations of model assumptions and the progression of the percentiles should be inspected visually, as well.

The predicted progression over age are displayed as lines and the manifest data as dots. Only three predictors were necessary to almost perfectly model the norm sample data with adjusted R2.

Sample Data

The package includes data from two large test norming projects, namely ELFE 1-6 (Lenhard & Schneider, 2006) and German adaption of the PPVT4 (A. Lenhard, Lenhard, Suggate & Seegerer, 2015), which can be used to run the analysis. Furthermore, large samples from the Center of Disease Control (CDC) on growth curves in childhood and adolescence (for computing Body Mass Index 'BMI' curves), life expectancy at birth and mortality per country from 1960 to 2017 (available from The World Bank). Type ?elfe, ?ppvt, ?CDC, ?mortality or ?life to display information on the data sets.

Terms of use / License

cNORM is licensed under GNU Affero General Public License v3 (AGPL-3.0). This means that copyrighted parts of cNORM can be used free of charge for commercial and non-commercial purposes that run under this same license, retain the copyright notice, provide their source code and correctly cite cNORM. Copyright protection includes, for example, the reproduction and distribution of source code or parts of the source code of cNORM or of graphics created with cNORM. The integration of the package into a server environment in order to access the functionality of the software (e.g. for online delivery of norm scores) is also subject to this license. However, a regression function determined with cNORM is not subject to copyright protection and may be used freely without preconditions. If you want to apply cNORM in a way that is not compatible with the terms of the AGPL 3.0 license, please do not hesitate to contact us to negotiate individual conditions.

If you want to use cNORM for scientific publications, we would also ask you to quote the source.

References

  • Lenhard, A., Lenhard, W., Segerer, R. & Suggate, S. (2015). Peabody Picture Vocabulary Test - Revision IV (Deutsche Adaption). Frankfurt a. M./Germany: Pearson Assessment.
  • Lenhard, A., Lenhard, W., Suggate, S. & Segerer, R. (2016). A continuous solution to the norming problem. Assessment, Online first, 1-14. doi: 10.1177/1073191116656437

News

cNORM:news and change-log

This file documents the development of the package as well as open issues or points for further improvements.

Version in 1.1.5

Date: 2019.02.06, preparing for CRAN release

Changes:

  • Cross Validation added to shiny GUI
  • cnorm.cv documentation improved
  • added information to BestModel output

Version in 1.1.4

Date: 2018.12.18

Changes:

  • scale parameter added to prepareData function
  • fix for plotNorm by group with missing values

Version in 1.1.3 - Second release on CRAN

Date: 2018.12.09

Changes:

  • rmarkdown moved from imports to suggests
  • cnorm.cv info added to README

Version in 1.1.2

Date: 2018.12.08

Changes:

  • deleted code in vignette needing to much build time
  • removed UTF-8 attributes from ppvt dataset and cleared all datasets from non ASCII signs
  • deleted code in vignette needing to much build time
  • additional tests run on R-hub
  • added rmarkdown to imports

Version in 1.1.1

Date: 2018.12.01

Changes:

  • Parameters added to cv.norm: Significance level for stratification process
  • Additional plot in cv.norm: delta R2 in norm score validation
  • Example in readme improved
  • CDC data: group variable set to center of interval
  • descend parameter removed from plotPercentileSeries, plotPercentiles, checkConsistency, rawTable & normTable; instead take default from model; vigniette updated accordingly
  • stop criterion added to data sampling in cnorm.cv
  • cv.norm: lines added to R2 delta plot
  • normTable and rawTable can now produce list of tables

Version in 1.1.0

Date: 2018.11.23

Changes:

  • Cross validation added: new function: cnorm.cv() for assessing RMSE for raw data and R2 and CROSSFIT for norm data
  • Data table output fpr cnorm.cv
  • rankBySlidingWindow now accessibe via prepareData()
  • group, raw, age and width can now be provided in cnorm.cv
  • parameter for full cross validation (seperate ranking for train and validation)
  • Additional NA checks and warning messages
  • plotPercentiles now with R2adjr in title

Version in 1.0.3

Date: 2018.11.16

Changes:

  • Additional instruction on series section of visualization tab in Shiny GUI
  • Code cleanup in bestModel function
  • SE added to plotNorm based on Oosterhuis, van der Ark & Sijtsma (2016)
  • RMSE added to model object (m$subsets), to plotRaw and to plotSubset
  • additional plotting options added to GUI:
    • plotting of differences in raw and norm plot
    • RMSE in model selection information function

Version in 1.0.2

Date: 2018.11.16

Changes:

  • Improvements in precision of plotPercentiles
  • error corrected in ppvt dataset: groups did not represent group means
  • function description in 'ranBySlidingWindow' updated
  • checking for missing packages in shiny GUI improved
  • user menu asking to install missing packages added
  • derive-function: more general approach with "order" parameter
  • plotDerivative function can now plot derivatives of higher order
  • exclude cases with missing values in rankByX functions
  • percentile columns added to rawTable and normTable
  • additional data cleansing for data objects imported from Excel file format

Version in 1.0.1 - First release on CRAN

Date: 2018.11.03

Changes:

  • Improvements in the GUI: Waiting circle shown to indicate ongoing computation
  • Additional help texts on best model in GUI
  • Additional plotting options in cNORM.GUI(): Raw Score and Norm Score plots
  • User input asking for missing suggested packages to install

Version in 1.0.0

Date: 2018.10.26

Changes:

  • Final polishing finished; realising first major version

Version in 0.9.20

Date: 2018.10.24

Changes:

  • GUI with Shiny finished
  • ... now working on finally releasing the package

Version in 0.9.19

Date: 2018.10.20

Changes:

  • API changed: predictNormValue renamed to predictNorm
  • Shiny GUI enhanced
  • Additional plotting options in plotNorm and plotRaw
  • less strict warning messages in predictNormValue function and checkConsistency

Version in 0.9.18

Date: 2018.10.08

Changes:

  • First shiny prototype (many thanks to Sebastian Gary); please use cNORM.GUI() to start user interface

Version in 0.9.17

Date: 2018.10.01

Changes:

  • predictNormValue fixed and optimized (many thanks to Sebastian Gary)
  • API change with respect to predictNormValue, rawTable and plotNorm
  • plotNorm: norm score boundaries guessed by min and max score from modelling

Version in 0.9.16

Date: 2018.09.21

Changes:

  • bug in predictNormValue partly fixed (further optimization necessary)
  • API change: plotValues renamed to plotRaw
  • new function: plotNorm

Version in 0.9.15

Date: 2018.09.18

Changes:

  • plotDensity function added
  • attributes added to data.frame to increase usability

Version in 0.9.13

Date: 2018.09.16

Changes:

  • predictNormValue with higher precision and effectivity
  • rawTableQuick removed from source code

Version in 0.9.12

Date: 2018.09.11

Changes:

  • 'simulateRasch' to simulate test data was added
  • old sim functions removed
  • documentation improved
  • new parameters to bestModel in order to force covariates into regression
  • additional checks in box cox functions

Version in 0.9.9

Date: 2018.09.06

Changes:

  • Enhancements to the 'prepareData' function

Version in 0.9.8

Date: 2018.09.05

Changes:

  • Life expectancy dataset of the World Bank added
  • Mortality of infants per 1000 life birth from 1960 to 2017 added
  • Minor chanes in functions to check data integrity and exceptions
  • Vignette updated

Version in 0.9.7

Date: 2018.08.31

Changes:

  • License changed to AGPL
  • Capitalizations in labels of plots
  • min and max renamed to minRaw and maxRaw (where appropriate)
  • terminology: standard or normal score instead of norm; score instead of value
  • new function for model validation: plotPercentileSeries
  • many functions now draw the default values from the model (plotting and predicting)

Version in 0.9.6

Date: 2018.08.28

Changes:

  • Minor improvements in function descriptions
  • API of plotSubset changed due to new plotting options

Version in 0.9.5

Date: 2018.08.25

Changes:

Version in 0.9.4

Date: 2018.08.23

Changes:

  • Generating group variable in rankBySlidingWindow
  • prarameters in plotPercentile to restrict age range
  • ppvt dataset restricted

Version in 0.9.3

Date: 2018.08.20

Changes:

  • plotNormCurves enhanced (Thanks to Sebastian Gary)
  • new function to plot semi parametric analyses via box cox power transformation: plotBoxCox
  • variable "explanatoryVariable" and "normVariable" in computePowers function renamed for easier API

Version in 0.9.2

Date: 2018.08.18

Changes:

  • Addeditional dataset: vocabulary development (PPVT4)

Version in 0.9.1

Date: 2018.08.16

Changes:

  • Added predictRawBC and predictNormBC for computing norm and raw values based on the parametric box cox power function parameters
  • New contributor: Sebastian Gary, welcome to the team!
  • Missing raw variable definition in plotValues corrected

Version in 0.9.0

Date: 2018.08.14

Changes:

  • Box Cox power transformation for regression model at specific age: optional parametric modelling for non-paramteric regression model

Version in 0.8.9

Date: 2018.08.13

Changes:

  • Convenience method for selection best model added: 'printSubset'
  • predictNormValue now accepts lists of values as well

Version in 0.8.8

Date: 2018.08.12

Changes:

  • parameter checks added
  • new parameter 'descriptives' addad to rankByGroup and rankBySlidingWindow added to retrieve descriptive statistics for each observation
  • improvements in the documentation
  • errors in bestModel and plotPercentiles corrected, when variable names are not as in example sample

Version in 0.8.6

Date: 2018.08.11

Changes:

  • new function: 'rankBySlidingWindow' which can be used for data sets with continuous age variables
  • error corrected for data being loaded from SPSS files
  • improvements in the documentation

Version in 0.8.5

Date: 2018.08.06

Changes:

  • new function for simulating data

Version in 0.8.3

Date: 2018.08.03

Changes:

  • Code cleaning and formatting

Version in 0.8.2

Date: 2018.08.02

Changes:

  • new internal function: rawTableQuick for speeding up generating norm tables Still has to be checked for working with descending values. Works only, if model assumptions are valid

Version in 0.8.0

Date: 2018.08.01

Changes:

  • new function: rawTable allows creating norm tables with assignment of raw -> norm values solves inverse function of regression model with brute force

Version in 0.7.11

Date: 2018.07.31

Changes:

  • improved 'prepareData' function

Version in 0.7.10

Date: 2018.07.28

Changes:

  • Description for computePowers improved
  • option in plotPercentile to use percentile scale or self defined c(mean, sd)

Version in 0.7.9

Date: 2018.07.28

Changes:

  • 'descend' parameter added to consistencyCheck and normTable
  • dependency rColorBrewer removed; plotPercentiles changed accordingly
  • latticeExtra moved to 'suggests'

Version in 0.7.8

Date: 2018.07.27

Changes:

  • 'descend' parameter added to consistencyCheck and normTable

Version in 0.7.7

Date: 2018.07.27

Changes:

  • Small changes to error messages in bestModel
  • printing of min value in plotDerivate removed

Version in 0.7.6

Date: 2018.07.27

Changes:

  • parameter 'predictors' added to allow self defined regression functions, e. g. for the inclusion of other ranking parameters like sex
  • 'type' parameter added to 'plotPercentile' to allow selection of quantile algorithm. Please consult help(quantile) for further information on 'type'

Version in 0.7.5

Date: 2018.07.26

Changes:

  • dependency dplyr removed: rankByOrder and plotPercentiles rewritten
  • API-change: derivationPlot renamed to plotDerivate
  • small changes to vignette and readme
  • parameter "raw" added to rankByGroup to specify raw value variable
  • rawVar and groupVar in plotPercentiles renamed to raw and group to make API more coherent

Version in 0.7.4

Date: 2018.07.25

Changes:

  • rankByOrder: ranking in descending order added

Version in 0.7.3

Date: 2018.07.25

Changes:

  • Additional ranking algorithms: Filliben, Levenbach, Yu & Huang; API changed to index
  • scale can be specified as double vector with c(mean, sd)
  • vignette updated accordingly

Version in 0.7.2

Date: 2018.07.24

Changes:

  • None. This is the first release

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("cNORM")

1.1.8 by Wolfgang Lenhard, 3 months ago


https://www.psychometrica.de/cNorm_en.html, https://github.com/WLenhard/cNORM


Report a bug at https://github.com/WLenhard/cNORM/issues


Browse source code at https://github.com/cran/cNORM


Authors: Wolfgang Lenhard [cre, aut] , Alexandra Lenhard [aut] , Sebastian Gary [ctb]


Documentation:   PDF Manual  


Task views: Psychometric Models and Methods


AGPL-3 license


Imports lattice, leaps, latticeExtra

Suggests knitr, markdown, shiny, shinycssloaders, foreign, readxl, rmarkdown


See at CRAN