```{r setup,include=F}
library(tidyverse)
library(quanteda)
library(quanteda.textmodels)
```

First, we load the [House of Commons Corpus from the ParlSpeech Dataset](https://dataverse.harvard.edu/file.xhtml?persistentId=doi:10.7910/DVN/L4OAKN/W2SVMF&version=1.0)

```{r}
Corp_HouseOfCommons_V2 <- readRDS("../data/Corp_HouseOfCommons_V2.rds")
```

We keep only speeches where the speaker is not the chair and is a member of the largest party.

We also transform the party variable into a factor and transform the date (including the creation of a year variable) since we will need both later.

```{r}
corp_hoc <- Corp_HouseOfCommons_V2 %>%
  filter(chair==F) %>%
  filter(party %in% c("Con","Lab")) %>%
  mutate(date=as.Date(date)) %>% 
  mutate(year=lubridate::year(date)) %>%
  mutate(party=as.factor(party))

corpus_hoc <- corpus(corp_hoc)
```

Overview: How much data do we have over time?

```{r}
corp_hoc %>% 
  ggplot()+
  geom_histogram(aes(x=date),stat="count")+
  facet_wrap(~party)
```

To make the task more manageable on a normal computer, we draw a sample of 4000 press releases per year. You may want to download this sample directly: [sample_corp.RData](http://theresagessler.eu/eui_cta/data/sample_corp.RData)


```{r}
sample_corp <- corpus(corp_hoc) %>% corpus_sample(4000,by="year")
#save(sample_corp,file="../data/sample_corp.RData")
```


If you downloaded the sample corpus, load it and start here.
Now, we extract data for one year from our dfm.


```{r}
load("../data/sample_corp.RData")

corpus_hoc_2019 <- corpus_subset(sample_corp,year==2019)

```

Now, we're ready to create a document feature matrix. You may choose to remove punctuation or do similar pre-processing.

Then, split the data into a test and a training set.

```{r}
dfm_hoc <- corpus_hoc_2019 %>% tokens() %>% dfm(remove_punct=T)

hoc_train <- dfm_sample(dfm_hoc,0.8*ndoc(dfm_hoc))
hoc_test <- dfm_subset(dfm_hoc,!(docnames(dfm_hoc) %in% docnames(hoc_train)))
```

We will talk about this again later but we may choose to remove features that are very rare or frequent in order to reduce the number of features.

```{r}
hoc_train <- dfm_trim(hoc_train,
                        min_docfreq=0.01,
                        max_docfreq=0.8,
                        docfreq_type="prop",
                        verbose=T)
```

Adjust the features of the test dfm to the training dfm

```{r}
hoc_test <- dfm_match(hoc_test,featnames(hoc_train))
```



It's time to train the model, predict & see how well we did.

```{r}
nb_model<-textmodel_nb(hoc_train,docvars(hoc_train,
  "party"))
nb_model

preds <- predict(nb_model,hoc_test)
table(preds,docvars(hoc_test,"party"))

# accuracy calculation

sum(docvars(hoc_test,"party")==preds)/length(preds)

# confusion matrix:
caret::confusionMatrix(preds,docvars(hoc_test,"party"))
```

## The effect of pre-processing

Repeat what we have done above but use different trimming thresholds - look at both the accuracy and the speed to get a feeling for the usefulness of pre-processing and the impact it may have on your predictions.

```{r}
#

```



## Implementing this at scale (advanced)

To really implement a substantive evaluation of accuracy metrics like Peterson and Spirling, we will have to repeat this procedure several times. For that, it is easiest to write a function and use a loop or an apply command.

Based on your previous code, write a function that calculates the accuracy for each year. 

Your function should definitely have a variable for the year but you may want to be able to vary other things - e.g. the trimming thresholds, the size of the training and test sample, ....

Tip: You can either calculate the accuracy yourself or, if you store the `caret::confusionMatrix()` output into a variable `confmat`, you can use `confmat$overall[['Accuracy']]` to directly get the number as an output.


```{r}

```

Now, run this function for each year.

```{r}

```