Session 7 R practice (offline)

Calculating quantities with text

Note: Due to missing packages in DataCamp light, namely quanteda, textdata, and tidytext, I have provided sample code that you can run on your own computer in RStudio. Make sure to run install.packages("quanteda"), install.packages("textdata"), and install.packages("tidytext") to install those packages if you don’t have them.

Each of the three exercises below can be run as standalone scripts, as they contain all needed imports within their code blocks

Exercise 6: Readability with Quanteda

How does the readability of JPMorgan’s annual report compare to the Citigroup annual report from class?

# load in readr (or tidyverse) to get read_file() function
library(readr)
# Load in all of JPM's 2014 annual report
doc <- read_file("https://rmc.link/Slides/acct420v2/Session_7/0000019617-14-000289.txt")
# Load in quanteda
library(quanteda)

Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     
Registered S3 method overwritten by 'dplyr':
  method           from
  print.rowwise_df     
Package version: 1.5.1
Parallel computing: 2 of 8 threads used.
See https://quanteda.io for tutorials and examples.

Attaching package: 㤼㸱quanteda㤼㸲

The following object is masked from 㤼㸱package:utils㤼㸲:

    View

# Calculate the three readability measures
textstat_readability(doc, "Flesch.Kincaid")

textstat_readability(doc, "FOG")

textstat_readability(doc, "Coleman.Liau")

#END

Exercise 7: Readability with Quanteda

How does the sentiment of JPMorgan’s annual report compare to the Citigroup annual report from class?

# load in readr (or tidyverse) to get read_file() function
library(readr)
# Load in all of JPM's 2014 annual report
doc <- read_file("https://rmc.link/Slides/acct420v2/Session_7/0000019617-14-000289.txt")
# Load in tidytext
library(tidytext)
# Load some components of tidyverse
library(dplyr)  # for the usual commands

Attaching package: 㤼㸱dplyr㤼㸲

The following objects are masked from 㤼㸱package:stats㤼㸲:

    filter, lag

The following objects are masked from 㤼㸱package:base㤼㸲:

    intersect, setdiff, setequal, union

library(tidyr)  # for spread
# convert document to tidy format
df_doc <- data.frame(ID=c("0000019617-14-000289"), text=c(doc),
                     stringsAsFactors = F) %>%
  unnest_tokens(word, text)
# Calculate term frequency
terms <- df_doc %>%
  count(ID, word, sort=TRUE) %>%
  ungroup()
total_terms <- terms %>% 
  group_by(ID) %>% 
  summarize(total = sum(n))
tf <- left_join(terms, total_terms) %>% mutate(tf=n/total)

Joining, by = "ID"

# Get the Loughran McDonald sentiment dictionary
sentiment <- get_sentiments("loughran")
# Merge in sentiment
tf_sent <- tf %>% left_join(sentiment)

Joining, by = "word"

# Calculate the three readability measures
tf_sent %>%
  spread(sentiment, tf, fill=0) %>%
  select(constraining, litigious, negative, positive, superfluous, uncertainty) %>%
  colSums()

constraining    litigious     negative     positive  superfluous  uncertainty 
0.0127076134 0.0178352469 0.0308215361 0.0055735147 0.0001672054 0.0220153829 

#END

Exercise 8: Make a word cloud after removing stopwords

# load in readr (or tidyverse) to get read_file() function
library(readr)
# Load in all of JPM's 2014 annual report
doc <- read_file("https://rmc.link/Slides/acct420v2/Session_7/0000019617-14-000289.txt")
# Load in quanteda and tidytext
library(quanteda)
library(tidytext)
# Load in some of tidyverse
library(dplyr)
# convert document to tidy format
df_doc <- data.frame(ID=c("0000019617-14-000289"), text=c(doc),
                     stringsAsFactors = F) %>%
  unnest_tokens(word, text)
# Pull a list of stopwords
stopwords <- stopwords(source="smart")
# Remove stopwords
df_doc_stop <- df_doc %>%
  anti_join(data.frame(word=stopwords, stringsAsFactors=F))

Joining, by = "word"

# Build a corpus object for quanteda
corp <- corpus(df_doc_stop, docid_field="ID", text_field="word")
# Plot a word cloud -- If you don't have RColorBrewer installed, you can
# remove the `color=` option.
textplot_wordcloud(dfm(corp), color = RColorBrewer::brewer.pal(9, "Set1"))

#END