Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("tokenizers")

0.3.0 by Lincoln Mullen, 2 years ago


https://docs.ropensci.org/tokenizers/, https://github.com/ropensci/tokenizers


Report a bug at https://github.com/ropensci/tokenizers/issues


Browse source code at https://github.com/cran/tokenizers


Authors: Lincoln Mullen [aut, cre] , Os Keyes [ctb] , Dmitriy Selivanov [ctb] , Jeffrey Arnold [ctb] , Kenneth Benoit [ctb]


Documentation:   PDF Manual  


MIT + file LICENSE license


Imports stringi, Rcpp, SnowballC

Suggests covr, knitr, rmarkdown, stopwords, testthat

Linking to Rcpp


Imported by DramaAnalysis, WhatsR, covfefe, deeplr, pdfsearch, proustr, rslp, textrecipes, tidypmc, tidytext, wactor.

Suggested by edgarWebR, torchdatasets.

Enhanced by quanteda.


See at CRAN