```{r setup,warning=F,message=F}
library(tidyverse)
library(quanteda)
library(quanteda.textmodels)
```

## Preparations

We now have a look at the EUI Thesis abstracts to evaluate what differentiates departments.

Load the data and create a corpus.

You can filter all texts below a certain number of characters by using `str_length()` or by checking if the abstract variable is NA - the data is a bit messy so I recommend this to get rid of data that was scraped incompletely or is not available in cadmus.

```{r}
load("../data/theses.RData")
```

```{r}
eui_corp <- theses %>%
  dplyr::filter(str_length(abstract)>300) %>% 
  corpus(text_field="abstract")

```

Now, create a dfm and do whatever pre-processing you think might be useful.

```{r}
eui_dfm <- tokens(eui_corp,
                  remove_punct=T,
                  remove_symbols=T,
                  split_hyphens=T) %>%
  dfm()

# Note that you could of course do more extensive preprocessing
# For example, as you may see later on, there are non-english theses in the sample
# hence, you might try to find and delete these documents using frequent french, german and italian words
```

## Training the classifier


Using the `department` document variable, train a Naive Bayes and a SVM model to predict the department.

If you want, skip the train-test split here because we don't need to see the accuracy of our model.

```{r}
model_nb <- textmodel_nb(eui_dfm,docvars(eui_dfm,"department"))
model_svm <- textmodel_svm(eui_dfm,docvars(eui_dfm,"department"))
```

## Feature evaluation

You can use `coef()` on the model to get a matrix of coefficients per feature and 'class'.

One thing that is a bit tricky is that both matrixes are different: for the svm coefficients, the departments will be the first dimension, for the naive bayes model, departments are the second dimension.

This is a bit tricky, so I include the code. I recommend

-  transposing the matrix of the svm coefficients using `t()`(that is, making the columns into rows and vice-versa) 
- storing both matrixes as a data frame
- storing the rownames in a new variable called feature

```{r}
coefs_svm <- coef(model_svm) %>%
  t() %>% 
  data.frame()%>%
  mutate(feature=rownames(.))

coefs_nb <- coef(model_nb) %>% 
  data.frame() %>%
  mutate(feature=rownames(.))
```

Now, try to find the most predictive features for two of the departments. You can do this for example by using `arrange()` on the dataframe - but there are other solutions, this is just a data management task.

You should see a difference between the results for both classifiers that is due to the way they reach their predictions building on the feature scores. 

```{r}
coefs_nb %>% 
  arrange(desc(eco_theses)) %>% 
  head() %>% 
  # no need for this line - it just makes the results nicer to read in rmarkdown
  knitr::kable(caption="NB Economy")

coefs_svm %>% 
  arrange(desc(eco_theses)) %>% 
  head() %>% 
  knitr::kable(caption="SVM Economy")


coefs_nb %>% 
  arrange(desc(sps_theses)) %>% 
  head() %>% 
  knitr::kable(caption="NB SPS")
coefs_svm %>% 
  arrange(desc(sps_theses)) %>% 
  head() %>% 
  knitr::kable(caption="SVM SPS")
```



Try a substantive evaluation: Think about words that stand for

- different methods
- different formats (cumulative vs. monograph)

Have a look at the coefficients of these words for each department. Again, there are several ways to do this - you just need to select the correct row of the data frame.

```{r}
# methods
coefs_nb[coefs_nb$feature=="data",]
coefs_nb[coefs_nb$feature=="regression",]
coefs_nb[coefs_nb$feature=="qualitative",]
coefs_nb[coefs_nb$feature=="narrative",]

# formats
coefs_nb[coefs_nb$feature=="chapter",]
coefs_nb[coefs_nb$feature=="article",]

coefs_nb %>% filter(feature=="article")
```

This was not part of your task - but one way to get the relation of the coefficients clearer would be to plot them. I do this here for the SVM coefficients since they are more intuitive and useful to understand.
However, the NB coefficients also become easier to interpret when you focus on the difference between the departments.


```{r}
coefs_svm %>%
  filter(feature %in% c("data","regression","qualitative","narrative",
                        "empirical","evidence","model", "measures")) %>%
  ggplot(aes(x=feature))+
  geom_point(aes(y=eco_theses,color="ECO"),alpha=0.7)+
  geom_point(aes(y=sps_theses,color="SPS"),alpha=0.7)+
  geom_point(aes(y=law_theses,color="LAW"),alpha=0.7)+
  geom_point(aes(y=hec_theses,color="HEC"),alpha=0.7)+
  ylab("coefficient")+
  geom_hline(aes(yintercept=0),linetype="dashed")+
  theme_minimal()+scale_color_discrete("Department")
```



Finally, think about keywords for your PhD: How predictive are they of each department? Do you fit where you are? :)


```{r}

coefs_nb[coefs_nb$feature=="migration",]
coefs_nb[coefs_nb$feature=="democracy",]
```

