Metrics for Assessing the Quality of Generated Text

Implementation of the BLEU-Score in 'C++' to evaluate the quality of generated text. The BLEU-Score, introduced by Papineni et al. (2002) , is a metric for evaluating the quality of generated text. It is based on the n-gram overlap between the generated text and reference texts. Additionally, the package provides some smoothing methods as described in Chen and Cherry (2014) .

sacRebleu

The goal of sacRebleu is to provide a simple interface to the BLEU score, a metric for evaluating the quality of generated text. This package is inspired by the NLTK and sacrebleu implementations of the BLEU score, and is implemented in C++ for the R programming language.

Installation

You can install the development version of sacRebleu from GitHub with:

Installation

# install.packages("devtools")
devtools::install_github("LazerLambda/sacRebleu")

Example

library(sacRebleu)
cand_corpus <- list("This is good", "This is not good")
ref_corpus <- list(list("Perfect outcome!", "Excellent!"), list("Not sufficient.", "Horrible."))
bleu_corpus <- bleu_corpus(ref_corpus, cand_corpus)