By creating crowd-sourcing tasks that can be easily posted and results retrieved using Amazon's Mechanical Turk (MTurk) API, researchers can use this solution to validate the quality of topics obtained from unsupervised or semi-supervised learning methods, and the relevance of topic labels assigned. This helps ensure that the topic modeling results are accurate and useful for research purposes. See Ying and others (2022)
Authors: Luwei Ying, Jacob Montgomery and Brandon Stewart
Please email all comments/questions to luwei.ying [AT] wustl.edu
The package is currently not available on CRAN. You can install the most recent development version using the devtools package. First you have to install devtools using the following code. Note that you only have to do this once
if(!require(devtools)) install.packages("devtools")
Then you'll need to install the dependency, "pyMTurkR", using the function install_github. If you need to interact with Mturk, be sure to follow the detailed instructions here: github.com/cloudyr/pyMTurkR
devtools::install_github("cloudyr/pyMTurkR")
Now you can install our package
devtools::install_github("Luwei-Ying/validateIt", dependencies=TRUE)
Note that this will install all the packages suggested and required to run our package. It may take a few minutes the first time, but this only needs to be done on the first use. In the future you can update to the most recent development version using the same code.