reproducer

Reproduce Statistical Analyses and Meta-Analyses

Includes data analysis functions (e.g., to calculate effect sizes and 95% Confidence Intervals (CI) on Standardised Effect Sizes (d) for ABBA cross-over repeated-measures experimental designs), data presentation functions (e.g., density curve overlaid on histogram), and the data sets analyzed in different research papers in software engineering (e.g., related to software defect prediction or multi-site experiment concerning the extent to which structured abstracts were clearer and more complete than conventional abstracts) to streamline reproducible research in software engineering.

The R package reproducer is aimed to support reproducible research in software engineering. See the package homepage for details and examples.

Installation

One may install the stable version from CRAN:

install.packages('reproducer', dependencies = TRUE)

You can use devtools to install the development version from my web site:

install.packages("devtools", dependencies = T, repos = "http://cran.r-project.org/")
library(devtools)
devtools::install_url("http://madeyski.e-informatyka.pl/download/R/reproducer_0.3.0.tar.gz")
library(reproducer)

Motivation

The motivation is to support reproducible research in software engineering via sharing data sets and code behind the published or just submitted papers.

News

CHANGES IN reproducer VERSION 0.3.0

NEW FEATURES

  • Functions including computational procedures used to reproduce the main findings, e.g., tables, in a joint paper: Barbara Kitchenham, Lech Madeyski, Pearl Brereton, "Meta-analysis for Families of Experiments in Software Engineering: A Systematic Review and Reproducibility and Validity Assessment": transformHgtoZr,
  • Function to search for industry relevant software projects available from GitHub related to a report: Lech Madeyski, “Training data preparation method,” tech. rep., code quest (research project NCBiR POIR.01.01.01-00-0792/16), 2019; as well as a paper: Tomasz Lewowski and Lech Madeyski, "Creating evolving project data sets in software engineering", 2019: searchForIndustryRelevantGitHubProjects
  • Data set: MadeyskiLewowski.IndustryRelevantGitHubJavaProjects20190324

UPDATED FEATURES

  • Updated function reproduceTablesOfPaperMetaAnalysisForFamiliesOfExperiments

CHANGES IN reproducer VERSION 0.2.1

UPDATED FEATURES

  • Fixed version of the package in README
  • Added warning to the ExtractMAStatistics function: it works with metafor version 2.0-0, but changes to metafor's method of providing access to its individual results may introduce errors into the function.

CHANGES IN reproducer VERSION 0.2.0

NEW FEATURES

  • functions including computational procedures used to reproduce the main findings, e.g., tables, in a joint paper: Barbara Kitchenham, Lech Madeyski, Pearl Brereton, "Meta-analysis for Families of Experiments: A Systematic Review and Reproducibility Assessment": calculateSmallSampleSizeAdjustment, constructEffectSizes, transformRtoZr, transformZrtoR, transformHgtoR, calculateHg, transformRtoHg, transformZrtoHgapprox, transformZrtoHg, PrepareForMetaAnalysisGtoR, ExtractMAStatistics, aggregateIndividualDocumentStatistics, reproduceTablesOfPaperMetaAnalysisForFamiliesOfExperiments.
  • unit tests
  • data sets used in the aforementioned paper: KitchenhamMadeyskiBrereton.MetaAnalysisReportedResults, KitchenhamMadeyskiBrereton.ABBAMetaAnalysisReportedResults, KitchenhamMadeyskiBrereton.ReportedEffectSizes, KitchenhamMadeyskiBrereton.ABBAReportedEffectSizes KitchenhamMadeyskiBrereton.ExpData, and KitchenhamMadeyskiBrereton.DocData

UPDATED FEATURES

CHANGES IN reproducer VERSION 0.1.9

NEW FEATURES

  • added unit tests and assertions (run-time tests)

UPDATED FEATURES

  • updated data set MadeyskiKitchenham.EUBASdata and functions getEffectSizesABBA, effectSizeCI

CHANGES IN reproducer VERSION 0.1.8

NEW FEATURES

  • added function getTheoreticalEffectSizeVariancesABBA

UPDATED FEATURES

  • updated functions getSimulationData, plotOutcomesForIndividualsInEachSequenceGroup, getEffectSizesABBA, effectSizeCI

CHANGES IN reproducer VERSION 0.1.7

NEW FEATURES

  • added function effectSizeCI to calculate 95% Confidence Intervals (CI) on Standardised Effect Sizes (d) for cross-over repeated-measures designs

REMOVED FEATURES

  • removed field dIG.Var.CurtinAve from the data frame returned by the reproduceSimulationResultsBasedOn500Reps1000Obs function (we agreed to write joint paper with Dr Curtin describing corrections to his equations to calculate effect size variances for continuous outcomes of cross-over clinical trials)

CHANGES IN reproducer VERSION 0.1.6

NEW FEATURES

  • added functions to reproduce simulation data, to calculate effect sizes and their variance discussed in a paper "Effect Sizes and their Variance for AB/BA Crossover Design Studies" by Lech Madeyski and Barbara Kitchenham.
    • getSimulationData
    • plotOutcomesForIndividualsInEachSequenceGroup
    • getEffectSizesABBA
    • getEffectSizesABBAIgnoringPeriodEffect
    • reproduceSimulationResultsBasedOn500Reps1000Obs
    • percentageInaccuracyOfLargeSampleVarianceApproximation
    • proportionOfSignificantTValuesUsingCorrectAnalysis
    • proportionOfSignificantTValuesUsingIncorrectAnalysis
  • new data set:
    • KitchenhamMadeyski.SimulatedCrossoverDataSets backed by functions (varianceSimulation, getSimulatedCrossoverDataSets) to reproduce the data set.

REMOVED FEATURES

  • removed function:
    • cloudOfWords

CHANGES IN reproducer VERSION 0.1.5

BUG FIXES

  • corrected typo in a surname of one of the data contributors
  • corrected the name of the university in Hong Kong
  • corrected the boxplotAndDensityCurveOnHistogram function
  • updated references to research papers supported by the package

CHANGES IN reproducer VERSION 0.1.4

NEW FEATURES

  • new data sets:
    • KitchenhamMadeyskiBudgen16.FINNISH
    • KitchenhamMadeyskiBudgen16.PolishSubjects
    • KitchenhamMadeyskiBudgen16.SubjectData
    • KitchenhamMadeyskiBudgen16.PolishData
    • KitchenhamMadeyskiBudgen16.DiffInDiffData
    • KitchenhamMadeyskiBudgen16.COCOMO
  • updated to be compatible with the ggplot2 2.0.0
  • minor visual changes in figures produced by the following functions:
    • densityCurveOnHistogram
    • boxplotHV
    • boxplotAndDensityCurveOnHistogram

CHANGES IN reproducer VERSION 0.1.3

NEW FEATURES

  • added general functions
    • printXTable
    • cloudOfWords
  • added functions to reproduce all of the tables, figures and outputs in the paper by Lech Madeyski and Barbara Kitchenham on how variations in experimental designs impact the construction of comparable effect sizes for meta-analysis:
    • reproduceForestPlotRandomEffects
    • reproduceMixedEffectsAnalysisWithEstimatedVarianceAndExperimentalDesignModerator
    • reproduceMixedEffectsAnalysisWithExperimentalDesignModerator
    • reproduceMixedEffectsForestPlotWithExperimentalDesignModerator
    • reproduceTableWithEffectSizesBasedOnMeanDifferences
    • reproduceTableWithPossibleModeratingFactors
    • reproduceTableWithSourceDataByCiolkowski
  • added data sets:
    • Ciolkowski09ESEM.MetaAnalysis.PBRvsCBRorAR
    • MadeyskiKitchenham.MetaAnalysis.PBRvsCBRorAR

BUG FIXES

  • corrected Madeyski15EISEJ.StudProjects$STUD data set

CHANGES IN reproducer VERSION 0.1.2

NEW FEATURES

  • The first published version of reproducer. It includes data sets:
    • Madeyski15SQJ.NDC
    • Madeyski15EISEJ.OpenProjects
    • Madeyski15EISEJ.PropProjects
    • Madeyski15EISEJ.StudProjects and functions (for importing data, visualization and descriptive analyses):
    • readExcelSheet
    • densityCurveOnHistogram
    • boxplotHV
    • boxplotAndDensityCurveOnHistogram

See the package homepage (http://madeyski.e-informatyka.pl/reproducible-research/) for documentation and examples.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.
install.packages("reproducer")

0.3.0 by Lech Madeyski, 2 months ago

http://madeyski.e-informatyka.pl/reproducible-research/

Browse source code at https://github.com/cran/reproducer

Authors: Lech Madeyski [cre, aut, ctb] , Barbara Kitchenham [ctb] (Data and code contributor) , Tomasz Lewowski [ctb] (Data and code contributor) , Marian Jureczko [ctb] (Data contributor) , David Budgen [ctb] (Data contributor) , Pearl Brereton [ctb] (Data contributor) , Jacky Keung [ctb] (Data contributor) , Stuart Charters [ctb] (Data contributor) , Shirley Gibbs [ctb] (Data contributor) , Amnart Pohthong [ctb] (Data contributor)

Documentation:   PDF Manual  

CC BY 4.0 license

Imports openxlsx, ggplot2, gridExtra, xtable, metafor, lme4, MASS, stats, reshape, GetoptLong, dplyr, httr, jsonlite, tidyr, readr, stringr, magrittr, tibble

Suggests testthat, assertthat

See at CRAN

R links

R homepage

Download R

Mailing lists

R documentation

R manuals

R FAQs

The R Journal

CRAN links

CRAN homepage

CRAN repository policy

Submit a package

METACRAN stuff

About METACRAN

At github

Report a bug