Proper Scoring Rules

Evaluating probabilistic forecasts via proper scoring rules. scoring implements the beta, power, and pseudospherical families of proper scoring rules, along with ordered versions of the latter two families. Included among these families are popular rules like the Brier (quadratic) score, logarithmic score, and spherical score. For two-alternative forecasts, also includes functionality for plotting scores that one would obtain under specific scoring rules.


News

Changes in Version 0.6

o Weighted Brier decompositions, optionally with resampling,
  are now available via brierscore().

o Scoring family parameters can now be specified by row,
  allowing for a unique scoring rule for each forecast.

o logscore() now receives the reverse argument so that
  scores go from -infty (worst) to 0 (best).

Changes in Version 0.5-1

o Fixed a bug in calcscore() when ordered=TRUE, whereby
  forecasts associated with outcomes in the final category
  were scored incorrectly.

Changes in Version 0.5

o Added support for scoring rules that are sensitive
  to distance; see Jose et al, 2009, Management Science.

o Added Brier score convenience function that also returns
  mean scores and Brier score decompositions.  Also added
  log score and spherical score convenience functions.

o Fixed bug in scalescores(), whereby a lower bound < 0 and
  reverse=TRUE resulted in incorrect scaling.

o Added code testing via testthat package to ensure bugs stay
  fixed.

Changes in Version 0.4

o Fixed bug in scalescores(), causing some min/max values of
  pow- and sph-families to be computed incorrectly.  This
  impacted attempted use of the 'bounds' argument.

o Corrected the 'made' column of WeatherProbs to reflect correct
  dates.

Changes in Version 0.3

o Added support for scoring forecasts with greater than 2
  alternatives.

o Added data WeatherProbs, which contains three-category weather
  forecasts concerning temperature and precipitation.

o Argument 'scaling' in calcscore() and plotscore() replaced with
  'bounds', so the user can supply desired lower and upper bounds
  of the scores.

o Added a 'reverse' argument that, if TRUE, assigns larger 
  scores to good forecasters (as opposed to smaller scores).

Changes in Version 0.2

o Fixed a bug associated with the beta family, where scaling=TRUE 
  sometimes still resulted in scores > 1.

o Updated Merkle & Steyvers reference to reflect its acceptance.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("scoring")

0.6 by Ed Merkle, a year ago


Browse source code at https://github.com/cran/scoring


Authors: Ed Merkle


Documentation:   PDF Manual  


GPL-2 license


Suggests testthat


Suggested by CalSim.


See at CRAN