Tools to create an interactive web-based visualization of a topic model that has been fit to a corpus of text data using Latent Dirichlet Allocation (LDA). Given the estimated parameters of the topic model, it computes various summary statistics as input to an interactive visualization built with D3.js that is accessed via a browser. The goal is to help users interpret the topics in their LDA topic model.
R package for interactive topic model visualization.
LDAvis is designed to help users interpret the topics in a topic model that has been fit to a corpus of text data. The package extracts information from a fitted LDA topic model to inform an interactive web-based visualization.
Once installed, we recommend a visit to the main help page:
library(LDAvis)help(createJSON, package = "LDAvis")
The documentation and example on the bottom of that page should provide a quick sense of how to create (and share) your own visualizations. If you want more details about the technical specifications of the visualization, see the vignette:
vignette("details", package = "LDAvis")
Note that LDAvis itself does not provide facilities for fitting the model (only visualizing a fitted model). If you want to perform LDA in R, there are several packages, including mallet, lda, and topicmodels.
If you want to perform LDA with the R package lda and visualize the result with LDAvis, our example of a 20-topic model fit to 2,000 movie reviews may be helpful.
LDAvis does not limit you to topic modeling facilities in R. If you use other tools (MALLET and gensim are popular), we recommend that you visit our Twenty Newsgroups example to help quickly understand what components LDAvis will need.
To share a visualization that you created using LDAvis, you can encode the state of the visualization into the URL by appending a string of the form:
to the end of the URL, where "k", "l", and "s" are strings indicating the desired values of the selected topic, the value of lambda, and the selected term, respectively. For more details, see the last section of our Movie Reviews example, or for a quick example, see the link here:
To read about the methodology behind LDAvis, see our paper, which we presented at the 2014 ACL Workshop on Interactive Language Learning, Visualization, and Interfaces in Baltimore on June 27, 2014.
We included one data set in LDAvis, 'TwentyNewsgroups', which consists of a list with 5 elements:
We also created a second data-only package called LDAvisData to hold additional example data sets. Currently there are three more examples available there:
CHANGES IN LDAvis VERSION 0.3.2
In some cases, the widths of the red topic-term bars did not decrease (as they should have) from term #1 to term #R under the relevance ranking with $\lambda = 1$. In other words, when $\lambda = 1$, there were topics in which a narrow red bar was displayed above a wider red bar, which should never happen. The issue had to do with the way topic-term bar widths are computed, and is discussed in detail in #32.
In the end, we implemented a quick fix in which we compute term frequencies implicitly, rather than using those supplied in the createJSON() function. The upside is that the red bar widths are now explicitly controlled to produce the correct visualization. The downside is that the blue bar widths do not necessarily match the user-supplied term frequencies exactly -- in fact, the new version of LDAvis ignores the user-supplied term frequencies entirely. In a few experiments, the differences are small, and decrease (as a proportion of the true term frequencies) as the true term frequencies increase. Thanks to @bmabey for leading us to this bug.
CHANGES IN LDAvis VERSION 0.3.1
CHANGES IN LDAvis VERSION 0.3
CHANGES IN LDAvis VERSION 0.2
The function runShiny was deprecated. Please use createJSON to create visualizations instead.
CHANGES IN LDAvis VERSION 0.1