```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE,
	cache=T)
library(quanteda)
library(readtext)
library(stringr)
library(dplyr)
library(httr)
library(stm)
```

We now apply the structural topic models to a real-world example: Parliamentary debates.

Topic models are often used in two different ways: (1) to determine broad topics in general corpuses, or (2) to determine 'framing' and different ways to speak about a topic / highlight aspects in thematically more narrow corpuses. We try to do the later:

We first fix the parliamentary debate corpus to be a bit more homogenous - we do this by using the agenda item variable and selecting only speeches about the European Union Withdrawal Bill.

We also discard the chair since its role is mostly procedural and (because we'll use the party variable later on) all speakers for whom party is missing.

Instead of doing this (which again needs some computational power), you can also load the pre-processed data from the [course webpage](http://theresagessler.eu/eui_cta/data/brexit.RData).


```{r,eval=F}
Corp_HouseOfCommons_V2 <- readRDS("../data/Corp_HouseOfCommons_V2.rds")
brexit_bill <- Corp_HouseOfCommons_V2 %>% 
  filter(agenda=="European Union (Withdrawal) Bill") %>% 
  filter(chair==F) %>%
  filter(is.na(party)==F)
save(brexit_bill,file="../data/brexit.RData")
```

## Preprocessing

Now, we create a corpus and do some pre-processing.


Generally, we can apply similar preprocessing steps as with all other models. However, [some studies have discouraged the use of extensive pre-processing for topic models](http://www.cs.cornell.edu/~xanda/winlp2017.pdf). However, there is no final word on this.

The reason I still do some pre-processing is pragmatic: Since topic models look for *clusters* of words, I recommend removing overly scarce terms - e.g. words that occur only a single time. But this primarily helps to reduce computing time. The same of course goes for punctuation.

Additionally, removing overly frequent terms and stopwords will improve the interpretability of the topic labels - so we also do that here. You may also want to use stemming but opinions on that differ.

```{r}

```


## Running a topic model

In the past, you had to convert a dfm into stm input using quanteda's `convert()` function. Now, stm can also directly work with dfms as input.

Sometimes, it is still preferable to convert a dfm into an stm explicitly so you know what happens - for example, stm removes documents with no features and you may not be able to use some of the evaluation functions if the number of documents differs between the original texts and your dfm.

However, you can also [remove empty documents from your dfm](https://github.com/quanteda/quanteda/issues/1647) to deal with this issue. I've copied (and commented out) two lines of code do this here.

```{r}
#keep <- ntoken(brexit_dfm) > 0
#brexit_dfm <- dfm_subset(brexit_dfm,keep)
```

Now, run the the actual model on this - you can choose the number of topics freely.

```{r}
stm_1 <- stm(brexit_dfm,K=20,seed = 2020,verbose=F)
save(stm_1,brexit_bill,keep,brexit_dfm,file="../data/stm_brexit1.RData")
```

If your model converges too slowly, load `stm_brexit1.RData` to continue.

## Interpretation

Now, have a look at the topics using `labelTopics()`. Do the topics make sense to you? Do you find the descriptors coherent? Which one do you find more easy to make sense of?

Also, pick a topic you find interesting and keep the number in mind.


```{r}


```

Plot the distribution of topics. Is your topic frequent in the corpus? If you find the plot difficult to understand, you can distribute the topics over several plots (by specifying `topics=1:n`) or adjust the canvas.


```{r}


```


Now, use `findThoughts()` to read some of the most characteristic documents for that topic. Do you feel fit the topic?

*You can increase the number of articles by setting a higher n.*


```{r}


```




## Incorporating meta information

The stm model allows you to incorporate meta information. This meta information can influence

a) how prevalent a topic is - this leads to smoother estimates and is often used when you want to estimate the effect of a variable
b) how the content of a topic varies across groups - this can be used to detect framing but is very sensitive

Pick a meta information from the data set that you think has an impact on the content of the speeches and use it as a prevalence covariate for a model.


If your model converges too slowly, load `stm_brexit2.RData` to continue.

```{r}



```

In a second step, look at the Topics and use `estimateEffect()` to estimate the effect of your prevalence covariate on a topic where you would think it may have an effect.


```{r}



```

You can see a model summary using `summary()`.   If you want to see the documentation for the summary command, check `?summary.estimateEffect()`:

```{r}


```


Alternatively, you can directly plot the results using `plot()`. If you want to see the documentation for the plot command, check `?plot.estimateEffect()`:

```{r}



```



## Using oolong

If you want to go a step further and get a feeling for the quality of the model, try the word and topic intrusion test in `oolong`.

You can read about the reasoning for both tests [here](https://papers.nips.cc/paper/2009/file/f92586a25bb3145facd64ab20fd554ff-Paper.pdf).

```{r}
install.packages("oolong")
library(oolong)
oolong_test <- create_oolong(stm_1)
oolong_test
oolong_test$do_word_intrusion_test()
oolong_test$lock()
oolong_test

oolong_test2 <- create_oolong(stm_1,brexit_bill$text)
oolong_test2
oolong_test2$do_topic_intrusion_test()
oolong_test2$lock()
#if you don't want to code all topics use this line:
# oolong_test2$lock(force=T)

```

Take the results with a grain of salt: We didn't really select a 'good' model, we just went with what we had. Still, it shows the challenges involved in itnerpreting topic models. Also, if your performance was low, think about whether pre-processing steps may have influenced the results (e.g. stemming) and that recognizing intruders is far more difficult when our goal is to measure different frames of the same debate, rather than broad topics.