---
title: "ACCT 420: Linear Regression"
author: 
  - name: Dr. Richard M. Crowley
    email: rcrowley@smu.edu.sg
    affiliation: https://rmc.link/
title-slide-attributes:
  data-background-video: "../Backgrounds/title.mp4"
  data-background-video-loop: "true"
  data-background-size: stretch
format:
  revealjs:
    theme: default
    css: quarto_white.css
    navigation-mode: vertical
    controls: true
    transition: slide
    background-transition: fade
    transition-speed: default
    width: 1536
    height: 864
    logo: ../Logos/logo2.png
    code-link: true
    code-line-numbers: false
    chalkboard:
      theme: whiteboard
    auto-stretch: false
revealjs-plugins:
  - attribution
execute:
  echo: true
  warning: false
  message: false
editor: source
---

# Front Matter {data-background-video="../Backgrounds/section.mp4"}

<!--# Note that the downlit and xml2 pacakges are required to compile -->
<!--# Installed extensions include `attribution`, `qrcode`, and `material-icons` -->

```{r helpers, echo=FALSE, warning=FALSE, message=F}
library(knitr)
library(kableExtra)

#' Pretty formatted HTML tables
#' 
#' @param df The dataframe to format as a table
#' @param names Names to use for columns if not using the df column names. Vector
#' @param highlight_cols Columns to highlight. Default is c(1) for highlighting the first row. Use c() to remove.
#' @param highlight_rows Rows to highlight. Default is c() to not highlight any rows.
#' @param color_rows Rows to highlight in color. Default is c() to not highlight any rows.
#' @param highlight_end_rows Rows to highlight. Uses a reverse indexing where 1 is the last row, 2 is second to last, etc.
#' @param full Use to span the full width of a slide. FALSE by default.
#' @param fixed_header Whether the first row should be treated as a header row. TRUE by default.
html_df <- function(df, names=NULL, highlight_cols=c(1), highlight_rows=c(), color_rows=c(), highlight_end_rows=c(), full=F, fixed_header=T) {
  if(!length(names)) {
    names=colnames(df)
  }
  kbl(df,"html", col.names = names, align = c("l",rep('c',length(cols)-1))) %>%
    kable_paper(c("striped","hover"), full_width=full, fixed_thead=F, html_font = "\"Source Sans Pro\", Helvetica, sans-serif") %>%
    {if(length(highlight_cols)) column_spec(., highlight_cols,bold=T, background = "#ffffff99") else .} %>%
    {if(length(highlight_rows)) row_spec(., highlight_rows,bold=T, background = "#ffffff99") else .} %>%
    {if(length(color_rows)) row_spec(., color_rows,bold=T, background = "#aaaaff99") else .} %>%
    {if(length(highlight_end_rows)) row_spec(., nrow(df) + 1 - highlight_end_rows,bold=T, background = "#ffffff99") else .}
}

library(tidyverse)
```

<!--# List packages here that are not being correctly highlighted -->
<!--# The usual cause is calling the library from a echo=FALSE cell -->

```{r}
#| include: false

# Note: currently broken in quarto-cli
library(downlit)
options(downlit.attached = "tidyverse")
```

<!--# Load in data for the lesson -->
```{r data, echo=FALSE, warning=FALSE, message=F}
# The data set is Compustat global, gind==602010 [note: Compustat changed this retroactively]; 
library(tidyverse)
df <- read.csv("../../Data/Session_2-1.csv", stringsAsFactors=FALSE)
df_fullyear <- df
df_full <- filter(df_fullyear, datadate < 20221231)
uol_fullyear <- filter(df, isin == "SG1S83002349")
uol <- filter(uol_fullyear, datadate < 20221231)
uol_answer = filter(df, isin == "SG1S83002349", datadate == 20221231)
#clean_df <- subset(df,fyear==2017 & !is.na(revt) & !is.na(ni) & revt > 1)
```

## Learning objectives {data-background="../Backgrounds/default.jpg"}

:::: columns

::: {.column width="50%"}

![](../Course_Diagrams/Map-2.png){fig-alt="Roadmap for the course"}

:::

::: {.column width="50%"}
- <b>Theory:</b>
    - Develop a logical approach to problem solving with data
        - Statistics
        - Causation
        - Hypothesis testing
- <b>Application:</b>
    - Predicting revenue for real estate firms
- <b>Methodology:</b>
    - Univariate stats
    - Linear regression
    - Visualization
:::

::::



## Datacamp {data-background="../Backgrounds/default.jpg"}

- For next week: 
   - 1 suggested chapters on tidyverse methods
- The full list of suggested Datacamp materials for the course is up on eLearn

> Datacamp is optional. If you find the coding difficult in today's lesson, you should go through the suggested datacamp chapters

## R Installation {data-background="../Backgrounds/coding.jpg"}

- If you haven't already, make sure to install R, R Studio, and Quarto!
    - Instructions are in Session 1's slides
    - You will need it for this week's assignment 
- Please install a few packages using the following code
    - These packages are also needed for the first assignment
    - You are welcome to explore other packages as well

```{r, eval=F}
# Run this in the R Console inside RStudio
install.packages(c("tidyverse","plotly"))
```

- Assignments will be provided as Quarto files

> The format will generally all be filled out -- you will just add to it, answer questions, analyze data, and explain your work.  Instructions and hints are in the same file

## Assignments for this course {data-background="../Backgrounds/reference.jpg" class="exclude"}

- Assignments will be posted online after the following lectures:

1. Session 2, on forecasting analytics
2. Session 4, on a mix of linear and logit models
3. Session 6, on forensic analytics
4. Session 9, on other methods

> For each assignment, you will have until 2 classes from release to complete it (2 weeks)

- If you are having trouble with the assignment, you can always book office hours.

## R Markdown: A quick guide {data-background="../Backgrounds/reference.jpg"}

- Headers and subheaders start with `#` and `##`, respectively
- Code blocks starts with ```` ```{r} ```` and end with ```` ``` ````
    - By default, all code and figures will show up in the document
- Inline code goes in a block starting with `` `r `` and ending with `` ` ``
- Italic font can be used by putting `*` or `_` around text
- Bold font can be used by putting `**` around text
    - E.g.: `**bold text**` becomes **bold text**
- To render the document, click <img src="../Figures/quarto_render.png" style="margin:0px">
- Math can be placed between `$` to use LaTeX notation
    - E.g. `$\frac{revt}{at}$` becomes $\frac{revt}{at}$
- Full equations (on their own line) can be placed between `$$`
- A block quote is prefixed with `>`
- For a complete guide, see the [Quarto tutorials](https://quarto.org/docs/get-started/hello/text-editor.html) or Datacamp's [Quarto Cheat Sheet](https://images.datacamp.com/image/upload/v1676540721/Marketing/Blog/Quarto_Cheat_Sheet.pdf)


# Application: Revenue prediction {data-background-video="../Backgrounds/section.mp4"}

## The question {data-background="../Backgrounds/default.jpg"}

> How can we predict revenue for a company, leveraging data about that company, related companies, and macro factors

<div class="fragment">

- Specific application: Real estate companies

![](../Figures/Singapore_Skyline_at_Bluehour.jpg){width="600px" fig-align="center"}

</div>

::: {.attribution}

Photo courtesy of [Wikimedia Commons](https://en.wikipedia.org/wiki/Central_Area,_Singapore#/media/File:Singapore_Skyline_at_Bluehour.jpg)

:::



## More specifically... {data-background="../Backgrounds/default.jpg"}

- Can we use a company's own accounting data to predict it's [future] revenue?
- Can we use other companies' accounting data to better predict all of their future revenue?
- Can we augment this data with macro economic data to further improve prediction?
    - Singapore business sentiment data

# Linear models {data-background-video="../Backgrounds/section.mp4"}

## What is a linear model? {data-background="../Backgrounds/default.jpg"}

$$
\hat{y}=\alpha + \beta \hat{x} + \varepsilon
$$

- The simplest model is trying to predict some outcome $\hat{y}$ as a function of an input $\hat{x}$
    - $\hat{y}$ in our case is a firm's revenue in a given year
    - $\hat{x}$ could be a firm's assets in a given year
    - $\alpha$ and $\beta$ are solved for
    - $\varepsilon$ is the error in the measurement

> I will refer to this as an *OLS* model -- **O**rdinary **L**east **S**quare regression

## Example {data-background="../Backgrounds/case.jpg"}

> Let's predict UOL's revenue for 2016

:::: {.columns}

::: {.column width="50%"}

::: {.r-stack}
![](../Figures/UOL1.jpg){width="400"}

![](../Figures/UOL2.jpg){.fragment width="400"}

![](../Figures/UOL3.jpg){.fragment width="400"}

![](../Figures/UOL5.jpg){.fragment width="500"}

![](../Figures/UOL7.jpg){.fragment width="600"}
:::

:::

::: {.column width="50%"}
- Compustat has data for them since 1989
    - Complete since 1994
        - Missing CapEx before that
```{r}
# revt: Revenue, at: Assets
summary(uol[,c("revt", "at")])
```
:::

::::


## Linear models in R {data-background="../Backgrounds/reference.jpg"}

- To run a linear model, use `lm()`
    - The first argument is a formula for your model, where `~` is used in place of an equals sign
        - The left side is what you want to predict
        - The right side is inputs for prediction, separated by `+`
    - The second argument is the data to use
- Additional variations for the formula:
    - Functions transforming inputs (as vectors), such as `log()`
    - Fully interacting variables using `*`
        - I.e., `A*B` includes: `A`, `B`, and `A times B` in the model
    - Interactions using `:`
        - I.e., `A:B` only includes `A times B` in the model

```{r, eval=F}
# Example:
lm(revt ~ at, data = uol)
```


## Example: UOL {data-background="../Backgrounds/case.jpg"}

```{r}
mod1 <- lm(revt ~ at, data = uol)
summary(mod1)
```

<div class="fragment exclude">

> \$1 more in assets leads to $0.12 more revenue

</div>

## Why is it called Ordinary Least *Squares*? {data-background="../Backgrounds/default.jpg"}

```{r}



```

```{r}
#| eval: true
#| echo: false
# Graph showing squared error
uolg <- uol[,c("at","revt")]
uolg$resid <- mod1$residuals
uolg$xleft <- ifelse(uolg$resid < 0,uolg$at,uolg$at - uolg$resid)
uolg$xright <- ifelse(uolg$resid < 0,uolg$at - uolg$resid, uol$at)
uolg$ytop <- ifelse(uolg$resid < 0,uolg$revt - uolg$resid,uol$revt)
uolg$ybottom <- ifelse(uolg$resid < 0,uolg$revt, uolg$revt - uolg$resid)
uolg$point <- TRUE

uolg2 <- uolg
uolg2$point <- FALSE
uolg2$at <- ifelse(uolg$resid < 0,uolg2$xright,uolg2$xleft)
uolg2$revt <- ifelse(uolg$resid < 0,uolg2$ytop,uolg2$ybottom)

uolg <- rbind(uolg, uolg2)
```

```{r}
#| eval: true
#| echo: true
#| fig-width: 16
#| fig-height: 2.25
#| fig-align: center

library(tidyverse)
# uolg is defined in the class session code file
uolg %>% ggplot(aes(y=revt, x=at)) + 
         geom_point(aes(shape=point, group=point)) + 
         scale_shape_manual(values=c(NA,18)) + 
         geom_smooth(method="lm", se=FALSE) +
         geom_errorbarh(aes(xmax=xright, xmin = xleft)) + 
         geom_errorbar(aes(ymax=ytop, ymin = ybottom)) + 
         theme(legend.position="none") + xlim(0, 20000) + ylim(0, 2500)
```

## Example: UOL {data-background="../Backgrounds/case.jpg"}

- This model wasn't so interesting...
    - Bigger firms have more revenue -- this is a given
    - Though it does tell us *something* about the relationship between assets and revenue

::: {.callout-important}
## Abstracting a problem

If we don't want to factor in firm size, we can use ratios to abstract away from it!

:::

- How about... revenue *growth*?
- Then we can use *change* in assets in the model
    - i.e., Asset growth

$$
\Delta x_t = \frac{x_t}{x_{t-1}} - 1
$$

## Calculating changes in R {data-background="../Backgrounds/reference.jpg"}

- The easiest way is using `{tidyverse}`'s `{dplyr}`
    - This has a `lag()` function
- The default way to do it is to create a vector manually

```{r, message=F}
# tidyverse
uol <- uol %>%
  mutate(revt_growth1 = revt / lag(revt) - 1)

# R way
uol$revt_growth2 = uol$revt / c(NA, uol$revt[-length(uol$revt)]) - 1

# Check that both ways are equivalent
identical(uol$revt_growth1, uol$revt_growth2)
```

> You can use whichever you are comfortable with

## A note on mutate() {data-background="../Backgrounds/reference.jpg"}

- `mutate()` adds variables to an existing data frame
    - If you need to manipulate a bunch of columns at once:
       - `across()` applies a transformation to multiple columns in a data frame
       - You can mix in `starts_with()` or `ends_with()` to pick columns by pattern
- Mutate can be very powerful when making complex sets of variables
    - Examples:
        - Calculating growth within company in a multi-company data frame
        - Normalizing data to be within a certain range for multiple variables at once

## Example: UOL with changes {data-background="../Backgrounds/case.jpg"}

```{r}
# Make the other needed change
uol <- uol %>%
  mutate(at_growth = at / lag(at) - 1) %>%  # Calculate asset growth
  rename(revt_growth = revt_growth1)        # Rename for readability

# Run the OLS model
mod2 <- lm(revt_growth ~ at_growth, data = uol)
summary(mod2)
```

## Example: UOL with changes {data-background="../Backgrounds/case.jpg"}

- $\Delta$Assets doesn't capture $\Delta$Revenue so well
- Perhaps change in total assets is a bad choice?
- Or perhaps we need to expand our model?

## Scaling up! {data-background="../Backgrounds/default.jpg"}

$$
\hat{y}=\alpha + \beta_1 \hat{x}_1 + \beta_2 \hat{x}_2 + \ldots + \varepsilon
$$

- OLS doesn't need to be restricted to just 1 input!
    - Not unlimited though (yet -- we'll get there)
        - Number of inputs must be less than the number of observations minus 1
- Each $\hat{x}_i$ is an input in our model
- Each $\beta_i$ is something we will solve for
- $\hat{y}$, $\alpha$, and $\varepsilon$ are the same as before

## Scaling up our model {data-background="../Backgrounds/case.jpg"}

> We have... 464 variables from Compustat Global alone!

- Let's just add them all?

<div class="fragment">
<br>

- We only have 28 observations...
    - 28 << 464...

> Now what?

</div>

## Scaling up our model {data-background="../Backgrounds/group.jpg"}

> Building a model requires careful thought!

- This is where having accounting and business knowledge comes in!

> What makes sense to add to our model?

## Brainstorming... {data-background="../Backgrounds/reference.jpg" class="present-only"}

<iframe allowfullscreen frameborder="0" height="720px" mozallowfullscreen src="https://app.wooclap.com/events/FJQORH/questions/6122323020028712447ba057" width="100%"></iframe>

## Scaling up our model {data-background="../Backgrounds/answer.jpg" class="exclude"}

- Some potential sources to consider:
    - Direct accounting relations
        - Financing and expenditures
    - Business management
        - Some management characteristics may matter
    - Economics
        - Macro econ: trade, economic growth, population, weather
        - Micro econ: Other related firms like suppliers and customers
    - Legal factors
        - Any changes in law?  Favorable or not?
    - Market factors
        - Interest rates, cost of capital, foreign exchange?

::: {.callout-tip appearance="simple"}

That's a lot!  How do we decide what to use though?

:::



## Scaling up our model {data-background="../Backgrounds/answer.jpg" class="exclude"}

- One possible improvement:

```{r}
# lct: short term liabilities, che: cash and equivalents, ebit: EBIT
uol <- uol %>%
  mutate(across(c(act, che, lct),
                   .fns = ~. / lag(.) - 1,
                   .names = paste0('{col}','_growth')))  # Calculate 3 growths
mod3 <- lm(revt_growth ~ act_growth + che_growth + lct_growth, data=uol)
summary(mod3)
```

## Practice: mutate() {data-background="../Backgrounds/coding.jpg"}

- This practice is to make sure you understand how to use mutate with lags
    - These are very important when dealing with business data!
- Do exercises 1 on today's R practice file:
    - <a href="https://rmc.link/420r2">R Practice</a>
    - Shortlink: <a href="https://rmc.link/420r2">rmc.link/420r2</a>

# Formalizing frequentist testing {data-background-video="../Backgrounds/section.mp4"}

## Why formalize? {data-background="../Backgrounds/default.jpg"}

- Our current approach has been ad hoc
    - What is our goal?
    - How will we know if we have achieved it?
- Formalization provides more rigor

## Scientific method {data-background="../Backgrounds/reference.jpg"}

1. Question
    - What are we trying to determine?
2. Hypothesis
    - What do we think will happen?  Build a model
3. Prediction
    - What exactly will we test?  Formalize model into a statistical approach
4. Testing
    - Test the model
5. Analysis
    - Did it work?

## Hypotheses {data-background="../Backgrounds/reference.jpg"}

- Null hypothesis, a.k.a. $H_0$
    - The status quo
    - Typically: The model *doesn't* work
- Alternative hypothesis, a.k.a. $H_1$ or $H_A$
    - The model *does* work (and perhaps how it works)
- Frequentist statistics can never directly support $H_0$!
    - Only can fail to find support for $H_A$
    - Even if our $p$-value is 1, we can't say that the results prove the null hypothesis!

> We will use test statistics to test the hypotheses

## Regression {data-background="../Backgrounds/default.jpg"}

- Regression (like OLS) has the following assumptions
    1. The data is generated following some model
        - E.g., a linear model
            - In two weeks, a logistic model
    2. The data conforms to some statistical properties as required by the test
    3. The model coefficients are something to precisely determine
        - I.e., the coefficients are constants
    4. $p$-values provide a measure of the chance of an error in a particular aspect of the model
        - For instance, the p-value on $\beta_1$ in $y=\alpha + \beta_1 x_1 + \varepsilon$ essentially gives the probability that the sign of $\beta_1$ is wrong

## OLS Statistical properties {data-background="../Backgrounds/default.jpg"}

$$
\begin{align*}
  \text{Theory:}\quad y &= \alpha + \beta_1 x_1 + \beta_2 x_2 + \ldots + \varepsilon \\
  \text{Data:}\quad \hat{y} &= \alpha + \beta_1 \hat{x}_1 + \beta_2 \hat{x}_2 + \ldots + \hat{\varepsilon}
\end{align*}
$$

1. There should be a *linear* relationship between $y$ and each $x_i$
    - I.e., $y$ is [approximated by] a constant multiple of each $x_i$
    - Otherwise we **shouldn't** use a *linear* regression
2. Each $\hat{x}_i$ is normally distributed
    - Not so important with larger data sets, but a good to adhere to
3. Each observation is independent
    - We'll violate this one for the sake of *causality*
4. Homoskedasticity: Variance in errors is constant
    - **This is important** for the tests' reliability
5. Not too much multicollinearity
    - Each $\hat{x}_i$ should be relatively independent from the others (some dependence is OK)

## Practical implications {data-background="../Backgrounds/default.jpg" class="present-only"}

> Models designed under a frequentist approach can only answer the question of "does this matter?"

- Is this a problem?

::: {.fragment .fade-in}

- Often this is enough
    - But we [generally] can't answer the question "is this irrelevant?"

:::

## Practical implications {data-background="../Backgrounds/default.jpg" class="present-not" visibility="hidden"}

> Models designed under a frequentist approach can only answer the question of "does this matter?"

- Is this a problem?

# Linear model implementation {data-background-video="../Backgrounds/section.mp4"}

## What exactly is a linear model? {data-background="../Backgrounds/default.jpg"}

- Anything OLS is linear
- Many transformations can be recast to linear
    - Ex.: $log(y) = \alpha + \beta_1 x_1 + \beta_2 x_2 + \beta_3 {x_1}^2 + \beta_4 x_1 \cdot x_2$
        - This is the same as $y' = \alpha + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \beta_4 x_4$ where:
            - $y' = log(y)$
            - $x_3 = {x_1}^2$
            - $x_4 = x_1 \cdot x_2$

> Linear models are *very* flexible

## Mental model of OLS: 1 input {data-background="../Backgrounds/default.jpg"}

![](../Figures/OLS_model.png){width="450px" fig-align="center"}

> Simple OLS measures a simple linear relationship between 1 input and 1 output

- E.g.: Our first regression this week: Revenue on assets

## Mental model of OLS: Multiple inputs {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}

> OLS measures simple linear relationships between a *set* of inputs and 1 output

- E.g.: This is what we did when scaling up earlier this session

:::

::: {.column width="50%"}
![](../Figures/OLS_model_m.png){width="450px" fig-align="center"}
:::

::::



## Other linear models: IV Regression (2SLS) {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}
> IV/2SLS models linear relationships where the effect of some $x_i$ on $y$ may be *confounded by outside factors*. 

- E.g.: Modeling the effect of management pay duration (like bond duration) on firms' choice to issue earnings forecasts
    - Instrument with CEO tenure (Cheng, Cho, and Kim 2015)

:::

::: {.column width="50%"}
![](../Figures/2SLS_model.png){width="450px" fig-align="center"}
:::

::::

::: aside

We won't use this in this course, but you should know it exists.

:::




## Other linear models: SUR {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}
> SUR models systems with *related error terms*

- E.g.: Modeling both revenue and earnings simultaneously

:::

::: {.column width="50%"}
![](../Figures/SUR_model.png){width="450px" fig-align="center"}
:::

::::

::: aside

We won't use this in this course, but you should know it exists.

:::


## Other linear models: 3SLS {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}
> 3SLS models systems of equations with *related outputs*

- E.g.: Modeling stock return, volatility, and volume simultaneously

:::

::: {.column width="50%"}
![](../Figures/3SLS_model.png){width="450px" fig-align="center"}
:::

::::

::: aside

We won't use this in this course, but you should know it exists.

:::





## Other linear models: SEM {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}
> SEM can model abstract and *multi-level relationships*

- E.g.: Showing that organizational commitment leads to higher job satisfaction, not the other way around (Poznanski and Bline 1999)

:::

::: {.column width="50%"}
![](../Figures/SEM_model.png){width="450px" fig-align="center"}
:::

::::

::: aside

We won't use this in this course, but you should know it exists.

:::

## Modeling choices: Model selection {data-background="../Backgrounds/default.jpg"}

> Pick what fits your problem!

- For forecasting a quantity:
    - Usually some sort of linear model regressed using OLS
    - The other model types mentioned are great for simultaneous forecasting of multiple outputs
- For forecasting a binary outcome:
    - Usually logit or a related model (we'll start this in 2 weeks)
- For forensics:
    - Usually logit or a related model

> There are many more model types though!

## Modeling choices: Variable selection {data-background="../Backgrounds/default.jpg"}

- The options:
    1. Use your own knowledge to select variables
    2. Use a selection model to automate it

:::: {.columns}

::: {.column width="50%"}
Own knowledge

- Build a model based on your knowledge of the problem and situation
- This is generally better
    - The result should be more interpretable
    - For prediction, you should know relationships better than most algorithms
:::

::: {.column width="50%"}
![](../Figures/brain.png){height="200px" fig-align="center"}
:::

::::


## Modeling choices: Automated selection {data-background="../Backgrounds/default.jpg"}

- Traditional methods include:
    - Forward selection: Start with nothing and add variables with the most contribution to Adj $R^2$ until it stops going up
    - Backward selection: Start with all inputs and remove variables with the worst (negative) contribution to Adj $R^2$ until it stops going up
    - Stepwise selection: Like forward selection, but drops non-significant predictors

:::: {.columns}

::: {.column width="50%"}
- Newer methods include:
    - Lasso/Elastic Net based models
        - Optimize with high penalties for complexity (i.e., # of inputs)
        - These are proven to be better
        - We will discuss these in week 6
:::

::: {.column width="50%"}
![](../Figures/artificial-neural-network-3501528_640.png){width="400px" fig-align="center"}
:::

::::



## The overfitting problem {data-background="../Backgrounds/default.jpg"}

> Or: Why do we like simpler models so much?

- Overfitting happens when a model fits in-sample data *too well*...
    - To the point where it also models any idiosyncrasies or errors in the data
    - This harms prediction performance
        - Directly harming our forecasts

> An overfitted model works really well on its own data, and quite poorly on new data

# Statistical tests and interpretation {data-background-video="../Backgrounds/section.mp4"}

## Coefficients {data-background="../Backgrounds/default.jpg"}

- In OLS: $\beta_i$
- A change in $x_i$ by 1 leads to a change in $y$ by $\beta_i$
- Essentially, the slope between $x$ and $y$
- The blue line in the chart is the regression line for $\hat{Revenue} = \alpha + \beta_i \hat{Assets}$ for all real estate firms globally, 1994-2021

```{r}
#| eval: true
#| echo: false
#| fig-width: 10
#| fig-height: 4
#| fig-align: center

df_full %>%
  ggplot(aes(y=revt, x=at)) + 
  geom_point() + 
  geom_smooth(method="lm") + 
  xlab("Assets") + 
  ylab("Revenue") +
  ggtitle('Revenue versus assets, globally, real estate firms')

```



## P-values {data-background="../Backgrounds/default.jpg"}

- $p$-values tell us the probability that an individual result is due to random chance

> "The P value is defined as the probability under the assumption of no effect or no difference (null hypothesis), of obtaining a result equal to or more extreme than what was actually observed." <br> -- Dahiru 2008

- These are very useful, particularly for a frequentist approach
- First used in the 1700s, but popularized by Ronald Fisher in the 1920s and 1930s

## P-values: Rule of thumb {data-background="../Backgrounds/reference.jpg"}

- If $p<0.05$ and the coefficient sign matches our mental model, we can consider this as supporting our model
    - If $p<0.05$ but the coefficient is opposite, then it is suggesting a problem with our model
    - If $p>0.10$, it is rejecting the alternative hypothesis
- If $0.05 < p < 0.10$ it depends...
    - For a small dataset or a complex problem, we can use $0.10$ as a cutoff
    - For a huge dataset or a simple problem, we should use $0.05$
        - We may even set a lower threshold if we have a ton of data

## One vs two tailed tests {data-background="../Backgrounds/default.jpg"}

- Best practice:  **Use a two tailed test** with a p-value cutoff of 0.05 or 0.10
    - 0.05 for easier problems, 0.10 for harder/noisier problems
- Second best practice: use a 1-tailed test with a p-value cutoff of **0.025 or 0.05**
        - This is mathematically equivalent to the best practice, but roundabout
- Common but generally inappropriate:
    - Use a one tailed test with cutoffs of 0.05 or 0.10 because your hypothesis is directional

```{r}
#| eval: true
#| echo: false
#| fig-width: 4
#| fig-height: 3
#| fig-align: center

plot_norm <- function(bound, outer) {
  x <- seq(-outer,outer,length=100)
  hx <- dnorm(x)
  plot(x, hx, type="n", xlab="z values", ylab="Normal PDF", main="Normal Distribution", axes=FALSE)
  lines(x, hx)
  i <- x < -bound
  polygon(c(-outer,x[i],-bound,-bound), c(0,hx[i],max(hx[i]),0), col="red")
  i <- x > bound
  polygon(c(bound, bound, x[i],outer), c(0,max(hx[i]), hx[i],0), col="red")
  axis(1, at=-outer:outer, pos=0)
}
plot_norm(1.96, 4)
```

## $R^2$ {data-background="../Backgrounds/default.jpg"}

- $R^2$ = Explained variation / Total variation
    - Variation = difference in the observed output variable from its own mean
- A high $R^2$ indicates that the model fits the data very well
- A low $R^2$ indicates that the model is missing much of the variation in the output
- $R^2$ is technically a *biased* estimator
- Adjusted $R^2$ downweights $R^2$ and makes it unbiased
    - $R^2_{Adj} = P R^2 + 1 - P$
        - Where $P=\frac{n-1}{n-p-1}$
        - $n$ is the number of observations
        - $p$ is the number of inputs in the model

## Test statistics {data-background="../Backgrounds/reference.jpg"}

- Testing a coefficient:
    - Use a $t$ or $z$ test
- Testing a model as a whole
    - $F$-test, check *adjusted* R squared as well
- Testing across models
    - Chi squared ($\chi^2$) test
    - Vuong test (comparing $R^2$)
    - <a href="https://en.wikipedia.org/wiki/Akaike_information_criterion">Akaike Information Criterion</a> (AIC) (Comparing MLEs, lower is better)

> All of these have p-values, except for AIC

## Confusion from frequentist approaches {data-background="../Backgrounds/default.jpg"}

- Possible contradictions:
    - $F$ test says the model is good yet nothing is statistically significant
    - Individual $p$-values are good yet the model isn't
    - One measure says the model is good yet another doesn't

> There are many ways to measure a model, each with their own merits.  They don't always agree, and it's on us to pick a reasonable measure.

# Causality {data-background-video="../Backgrounds/section.mp4"}

## What is causality? {data-background="../Backgrounds/default.jpg"}

$A \rightarrow B$

- Causality is $A$ *causing* $B$
    - This means more than $A$ and $B$ are correlated
- I.e., If $A$ changes, $B$ changes.  But $B$ changing doesn't mean $A$ changed
    - Unless $B$ is 100% driven by $A$
- Very difficult to determine, particularly for events that happen [almost] simultaneously
- [Examples of correlations that aren't causation](http://tylervigen.com/spurious-correlations)

[![](https://imgs.xkcd.com/comics/correlation.png){height="200px" fig-align="center"}](https://xkcd.com/552/)


## Time and causality {data-background="../Backgrounds/default.jpg"}

$$
A \rightarrow B \text{ or } A \leftarrow B ?\\
A_t \rightarrow B_{t+1}
$$

- If there is a separation in time, it's easier to say $A$ caused $B$
    - Observe $A$, then see if $B$ changes after
- Conveniently, we have this structure when forecasting
    - Consider a model like:

$$
Revenue_{t+1} = Revenue_t + \ldots
$$

> It would be quite difficult for $Revenue_{t+1}$ to cause $Revenue_t$

## Time and causality break down {data-background="../Backgrounds/default.jpg"}

$$
A_t \rightarrow B_{t+1}? \quad \text{OR} \quad C \rightarrow A_t\text{ and }C \rightarrow B_{t+1}?
$$

- The above illustrates the *Correlated omitted variable problem*
    - $A$ doesn't cause $B$...  Instead, some other force $C$ causes both
    - This is the bane of social scientists everywhere
- It is less important for *predictive* analytics, as we care more about performance, but...
    - It can complicate interpreting your results
    - Figuring out $C$ can help improve you model's predictions (So find C!)

[![](https://imgs.xkcd.com/comics/cell_phones.png){height="250px" fig-align="center"}](https://xkcd.com/925/)

# Revisiting the previous problem {data-background-video="../Backgrounds/section.mp4"}

## Formalizing our last test {data-background="../Backgrounds/question.jpg"}

<div class="alignleft">

1. Question
    - $~$
2. Hypotheses
    - $H_0$: 
    - $H_1$: 
3. Prediction
    - $~$
4. Testing:
    - $~$
5. Statistical tests:
    - Individual variables:
    - Model:
</div>

## Formalizing our last test {data-background="../Backgrounds/answer.jpg" class="exclude"}

1. Question
    - Can we predict changes in revenue using a firm's accounting information?
2. Hypotheses
    - $H_0$: Our variables do not predict UOL's change in revenue
    - $H_1$: Our variables help to predict UOL's change in revenue
3. Prediction
    - Use an OLS model
4. Same results as before
5. Statistical tests:
    - Individual variables
        - Growth in current liabilities loads (+) at $p<0.01$ using a t-test
        - Growth in EBIT loads (+) at $p<0.10$ using a t-test
    - Model: F-test is significant at $p<0.05$

## Is this model better? {data-background="../Backgrounds/default.jpg"}

```{r}
anova(mod2, mod3, test="Chisq")
```

> A bit better at $p<0.05$

- This means our model with change in current liabilities, cash, and EBIT appears to be better than the model with change in assets.

# Scaling up {data-background-video="../Backgrounds/section.mp4"}

## Expanding our methodology {data-background="../Backgrounds/default.jpg"}

- Why should we limit ourselves to 1 firm's data?
- The nature of data analysis is such:

> Adding more data usually helps improve predictions

- Assuming:
    - The data isn't of low quality (too noisy)
    - The data is relevant
    - Any differences can be reasonably controlled for

## Fine tuning our question {data-background="../Backgrounds/default.jpg"}

- Previously: Can we predict revenue using a firm's accounting information?

::: {.callout-warning}
## Problems with our original question

1. We were using simultaneous $\hat{Y}$ and $\hat{X}$ variables
    - Thus, it was not forecasting
2. Simultaneous accounting data is very dependent/correlated
    - We were violating OLS regression assumptions

:::

- Now: Can we predict *future* revenue using a firm's accounting information?
    - What do we need to change? $\hat{y}$ will need to be 1 year in the future

::: {.callout-tip}
## What this revised question does better

1. It is a proper prediction problem
    - We are using old data to predict a new outcome
2. We don't need to worry much about dependence

:::

## First things first {data-background="../Backgrounds/default.jpg"}

- When using a lot of data, it is important to make sure the data is clean
- In our case, we may want to remove any very small firms

```{r}
# Ensure firms have at least $1M (local currency), and have revenue
# df_full contains all real estate companies excluding North America
df_clean <- df_full %>%
  filter(at>1, revt>0)

# We cleaned out 596 observations!
print(c(nrow(df_full), nrow(df_clean)))

# Another useful cleaning function:
# Replaces NaN, Inf, and -Inf with NA for all numeric variables in the data!
df_clean <- df_clean %>%
  mutate(across(where(is.numeric), ~replace(., !is.finite(.), NA)))
```


## Looking back at the prior models {data-background="../Backgrounds/case.jpg"}

```{r, message=F, warning=F}
uol <- uol %>% mutate(revt_lead = lead(revt))  # From dplyr
forecast1 <- lm(revt_lead ~ act + che + lct, data=uol)
library(broom)  # Lets us view bigger regression outputs in a tidy fashion
tidy(forecast1)  # Present regression output
glance(forecast1)  # Present regression statistics
```

> This model is ok, but we can do better.

## Expanding the prior model {data-background="../Backgrounds/case.jpg"}

```{r}
forecast2 <- 
  lm(revt_lead ~ revt + act + che + lct + dp + ebit , data=uol)
tidy(forecast2)
```

- Revenue to capture stickiness of revenue
- Current assets & Cash (and equivalents) to capture asset base
- Current liabilities to capture payments due
- Depreciation to capture decrease in real estate asset values
- EBIT to capture operational performance

## Expanding the prior model {data-background="../Backgrounds/case.jpg"}

```{r}
glance(forecast2)

anova(forecast1, forecast2, test="Chisq")
```

> This is better (Adj. $R^2$, $\chi^2$, AIC).

## All Singapore real estate companies {data-background="../Backgrounds/case.jpg"}

```{r}
# Note the group_by -- without it, lead() will pull from the subsequent firm!
# ungroup() tells R that we finished grouping
df_clean <- df_clean %>% 
  group_by(isin) %>% 
  mutate(revt_lead = lead(revt)) %>%
  ungroup()
```

![](../Figures/Grouping.png){height="400px" fig-align="center"}

## All Singapore real estate companies {data-background="../Backgrounds/case.jpg"}

```{r}
forecast3 <-
  lm(revt_lead ~ revt + act + che + lct + dp + ebit,
     data=df_clean[df_clean$fic=="SGP",])
tidy(forecast3)
```

## All Singapore real estate companies {data-background="../Backgrounds/case.jpg"}

```{r}
glance(forecast3)
```

> Lower adjusted $R^2$ -- This is worse?  Why?

- Note: $\chi^2$ can only be used for models on the same data
    - Same for AIC

## Worldwide real estate companies {data-background="../Backgrounds/case.jpg"}

```{r}
forecast4 <-
  lm(revt_lead ~ revt + act + che + lct + dp + ebit , data=df_clean)
tidy(forecast4)
```

## Worldwide real estate companies {data-background="../Backgrounds/case.jpg"}

```{r}
glance(forecast4)
```

> Higher adjusted $R^2$ -- better!

- Note: $\chi^2$ can only be used for models on the same data
    - Same for AIC

## Model accuracy {data-background="../Backgrounds/question.jpg"}

> Why is the UOL model better than the Singapore model?

- Ranking:
    1. Worldwide real estate model
    2. UOL model
    3. Singapore real estate model

> <span class="fragment exclude">Different sources of noise, amounts of data</span>

## Practice: group_by() {data-background="../Backgrounds/coding.jpg"}

- This practice is to make sure you understand how to use mutate with leads and lags when there are multiple companies in the data
    - We'll almost always work with multiple companies!
- Do exercises 2 and 3 on today's R practice file:
    - <a href="https://rmc.link/420r2">R Practice</a>
    - Shortlink: <a href="https://rmc.link/420r2">rmc.link/420r2</a>


# Expanding our problem with macro data {data-background-video="../Backgrounds/section.mp4"}

## Macro data sources {data-background="../Backgrounds/default.jpg"}

- For Singapore: <a href="https://data.gov.sg">Data.gov.sg</a>
    - Covers: Economy, education, environment, finance, health, infrastructure, society, technology, transport
- For real estate in Singapore: URA's REALIS system
    - Access through the library
- WRDS has some as well
- For US: <a href="https://data.gov">data.gov</a>, as well as many agency websites
    - Like <a href="https://www.bls.gov/data/">BLS</a> or the <a href="https://fred.stlouisfed.org/">Federal Reserve</a>

<img src="../Figures/WRDS.png" height="85px"> <img src="../Figures/dataSG.svg" height="85px"> <img src="../Figures/dataUS.png" height="85px">

## Our Macro data {data-background="../Backgrounds/default.jpg" class="small"}

```{r, include=F}
library(DT)
options(DT.options = list(scrollY="400px"))
```

```{r}
#| eval: true
#| echo: true
#| warning: false
#| error: false

library(tidyverse)
# First column, first 10 rows...
read_csv("../../Data/Session_2-Macro.csv") %>%
  .[1:10, 1] %>%
  DT::datatable()
```

## Our macro data {data-background="../Backgrounds/default.jpg" class="small"}

```{r, include=F}
library(DT)
options(DT.options = list(scrollY="400px"))
```

```{r}
#| eval: true
#| echo: true
#| warning: false
#| error: false

# Skip the header
# Next 10 rows and columns
read_csv("../../Data/Session_2-Macro.csv", skip=10) %>%
  .[1:10, 1:10] %>%
  DT::datatable()
```

## Panel data {data-background="../Backgrounds/reference.jpg"}

- Panel data refers to data with the following characteristics:
    - There is a time dimension
    - There is at least 1 other dimension to the data (firm, country, etc.)
- Special cases:
    - A panel where all dimensions have the same number of observations is called *balanced*
        - Otherwise we call it *unbalanced*
    - A panel missing the time dimension is *cross-sectional*
    - A panel missing the other dimension(s) is a *time series*
- Format:
    - Long: Indexed by all dimensions
    - Wide: Indexed only by some dimensions

## Panel data {data-background="../Backgrounds/reference.jpg"}

![](../Figures/Panel_types.png){width="50%" fig-align="center"}

> Data frames are usually wide panels




## Loading macro data {data-background="../Backgrounds/default.jpg"}

- Singapore business expectations data (from [SingStat](https://tablebuilder.singstat.gov.sg/table/TS/M250431)

```{r, echo=T, message=F, warning=F}
expectations <- read_csv("../../Data/Session_2-Macro.csv",
                         skip=10, na="na") %>%          # Needed to load file
  filter(row_number() < 21) %>%                         # Drop the footer
  rename(industry=`Data Series`) %>%                    # Rename column
  pivot_longer(!industry, names_to='yearQ',
               values_to='fin_sentiment') %>%           # Cast wide to long
  mutate(year = as.numeric(substr(yearQ, 1, 4))) %>%    # split out year
  mutate(quarter = as.numeric(substr(yearQ, 6, 6))) %>% # split out quarter
  select(-yearQ)                                        # Remove measure

# extract out Q1, finance only
expectations_re <- expectations %>%
  filter(quarter == 1,                               # Keep only the Q1
         industry == "Real Estate")                  # Keep only real estate
```

::: {.callout-tip}
## Casting between data frame shapes

The `pivot_wider()` and `pivot_longer()` functions work well. See the `{dplyr}` documentation for more details. In the code above, the first argument is the columns to turn into rows. `!industry` means all columns except `industry`. `names_to` (`value_to`) specifies the variable name to contain the column names (data) after transforming to long.



:::

## What was in the macro data? {data-background="../Backgrounds/default.jpg" class="small"}


```{r, warning=F}
expectations %>%
  arrange(industry, year, quarter) %>%  # sort the data
  filter(year == 2022) %>%
  DT::datatable(rownames=FALSE)  # display using DT
```

## dplyr makes merging easy {data-background="../Backgrounds/reference.jpg"}

- For merging, use `{dplyr}`'s `*_join()` commands
    - `left_join()` for merging a dataset into another
    - `inner_join()` for keeping only matched observations
    - `outer_join()` for making all possible combinations
- For sorting, `{dplyr}`'s `arrange()` command is easy to use
    - For sorting in reverse, combine `arrange()` with `desc()`
        - Or you can just put a `-` in front of the column name

## Merging example {data-background="../Backgrounds/default.jpg"}

> Merge in the finance sentiment data to our accounting data

```{r}
# subset out our Singaporean data, since our macro data is Singapore-specific
df_SG <- df_clean %>% filter(fic == "SGP")

# Create year in df_SG (date is given by datadate as YYYYMMDD)
df_SG$year = round(df_SG$datadate / 10000, digits=0)

# Combine datasets
# Notice how it automatically figures out to join by "year"
df_SG_macro <- left_join(df_SG, expectations_re[,c("year","fin_sentiment")])
```

# Predicting with macro data {data-background-video="../Backgrounds/section.mp4"}

## Building in macro data {data-background="../Backgrounds/case.jpg"}

- First try: Just add it in

```{r}
macro1 <- lm(revt_lead ~ revt + act + che + lct + dp + ebit + fin_sentiment,
             data=df_SG_macro)
library(broom)
tidy(macro1)
```

> It isn't significant.  Why is this?

## Brainstorming... {data-background="../Backgrounds/rain.jpg" class="default present-only"}

<iframe allowfullscreen frameborder="0" height="660px" mozallowfullscreen src="https://app.wooclap.com/events/GBJOST/questions/5f530b02008031118f39b6d1" width="100%"></iframe>

## Brainstorming... {data-background="../Backgrounds/question.jpg" class="default present-not" visibility="hidden"}

> Why isn't the macro data significant?

## Scaling matters {data-background="../Backgrounds/default.jpg"}

- All of our firm data is on the same terms as revenue: dollars within a given firm
- But `fin_sentiment` is a constant scale...
    - We need to scale this to fit the problem
         - The current scale would work for revenue growth

:::: {.columns}

::: {.column width="50%"}
```{r}
#| eval: true
#| echo: true
#| fig-width: 6
#| fig-height: 3
#| fig-align: center

df_SG_macro %>%
  ggplot(aes(y=revt_lead,
             x=fin_sentiment)) +
  geom_point()
```
:::

::: {.column width="50%"}
```{r}
#| eval: true
#| echo: true
#| fig-width: 6
#| fig-height: 3
#| fig-align: center

df_SG_macro %>%
  ggplot(aes(y=revt_lead,
             x=fin_sentiment * revt)) +
  geom_point()
```
:::

::::


## Scaled macro data {data-background="../Backgrounds/case.jpg"}

- Scale by revenue

```{r}
macro3 <-
  lm(revt_lead ~ revt + act + che + lct + dp + ebit + fin_sentiment:revt,
     data=df_SG_macro)
tidy(macro3)
glance(macro3)
```

## Model comparisons {data-background="../Backgrounds/case.jpg"}

```{r}
# Ensure that we use the same data (fin_sentiment is missing in 1994)
baseline <-
  lm(revt_lead ~ revt + act + che + lct + dp + ebit,
     data=df_SG_macro[!is.na(df_SG_macro$fin_sentiment),])
glance(baseline)
glance(macro3)
```

> Adjusted $R^2$ and AIC are slightly better with macro data

## Model comparisons {data-background="../Backgrounds/case.jpg"}

```{r}
anova(baseline, macro3, test="Chisq")
```

> Macro model definitely fits better than the baseline model!

## Takeaway {data-background="../Backgrounds/reference.jpg"}

```{r, include=F, echo=F}
revenue_mean = mean(df_SG_macro$revt, na.rm=T)
r_sd <- round(sd(df_SG_macro$fin_sentiment, na.rm=T),1)
r_min <- min(df_SG_macro$fin_sentiment, na.rm=T)
r_max <- max(df_SG_macro$fin_sentiment, na.rm=T)
rev_base <- macro3$coefficients[["revt"]]
rev <- macro3$coefficients[["revt:fin_sentiment"]]
r_rev = round(100 * r_sd * rev,2)
r_rev_base = round(100 * rev_base, 2)
rev_min <- round((r_min * rev)*100,2)
rev_max <- round((r_max * rev)*100,2)
```

1. Adding macro data can help explain some exogenous variation in a model
    - Exogenous meaning outside of the firms, in this case
2. Scaling is very important
    - Not scaling properly can suppress some effects from being visible

<div class="fragment">
> Interpretating the macro variable

- All else equal, the average firm has revenue stickiness of `r r_rev_base`%
- For every 1 S.D. increase in `fin_sentiment` (`r r_sd` points)
    - Revenue stickiness changes by ~`r r_rev`%
- Over the range of sentiment data (`r r_min` to `r r_max`)... 
    - Revenue stickiness changes from +`r rev_min`% to `r rev_max`%

</div>

## Scaling up our model, again {data-background="../Backgrounds/group.jpg"}

> Building a model requires careful thought!

- What macroeconomic data makes sense to add to our model?

> This is where having accounting and business knowledge comes in!

## Brainstorming... {data-background="../Backgrounds/rain.jpg" class="present-only"}

<iframe allowfullscreen frameborder="0" height="660px" mozallowfullscreen src="https://app.wooclap.com/events/GBJOST/questions/5f530b15500d2726de6ecdec" width="100%"></iframe>

## Brainstorming... {data-background="../Backgrounds/group.jpg" class="present-not" visibility="hidden"}

> What other macro data would you like to add to this model?

# Validation: Is it better? {data-background-video="../Backgrounds/section.mp4"}

## Validation {data-background="../Backgrounds/default.jpg" class="default"}

- Ideal:
    - Withhold the last year (or a few) of data when building the model
    - Check performance on *hold out sample*
    - This is *out of sample* testing
- Sometimes acceptable:
    - Withhold a random sample of data when building the model
    - Check performance on *hold out sample*

## Estimation {data-background="../Backgrounds/case.jpg" class="default"}

- As we never constructed a hold out sample, let's end by estimating UOL's *2022* year revenue
    - Announced in 2023

```{r}
p_uol <- predict(forecast2, uol[uol$fyear==2021,])
p_base <- predict(baseline,
  df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear==2021,])
p_macro <- predict(macro3,
  df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear==2021,])
p_world <- predict(forecast4,
  df_clean[df_clean$isin=="SG1S83002349" & df_clean$fyear==2021,])
preds <- c(p_uol, p_base, p_macro, p_world)
names(preds) <- c("UOL 2022 UOL", "UOL 2022 Base", "UOL 2022 Macro",
                  "UOL 2022 World")
preds
```

## Visualizing our prediction {data-background="../Backgrounds/case.jpg" class="default"}

```{r}
#| eval: true
#| echo: false
#| fig-width: 15.5
#| fig-height: 7.5
#| fig-align: center

library(plotly)
df_SG_macro$pred_base <- predict(baseline, df_SG_macro)
df_SG_macro$pred_macro <- predict(macro3, df_SG_macro)
df_clean$pred_world <- predict(forecast4, df_clean)
uol$pred_uol <- predict(forecast2, uol)
df_preds <- data.frame(preds=preds, fyear=c(2022,2022,2022,2022), model=c("UOL only", "Base", "Macro", "World"))
plot <- ggplot() + 
  geom_point(data=df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,], aes(y=revt_lead,x=fyear+1, color="Actual")) +
  geom_line(data=df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,], aes(y=revt_lead,x=fyear+1, color="Actual")) + 
  geom_point(data=uol[uol$fyear < 2021,], aes(y=pred_uol,x=fyear+1, color="UOL only")) +
  geom_line(data=uol[uol$fyear < 2021,], aes(y=pred_uol,x=fyear+1, color="UOL only")) +
  geom_point(data=df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,], aes(y=pred_base,x=fyear+1, color="Base")) +
  geom_line(data=df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,], aes(y=pred_base,x=fyear+1, color="Base")) +
  geom_point(data=df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,], aes(y=pred_macro,x=fyear+1, color="Macro")) +
  geom_line(data=df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,], aes(y=pred_macro,x=fyear+1, color="Macro")) + 
  geom_point(data=df_clean[df_clean$isin=="SG1S83002349" & df_clean$fyear < 2021,], aes(y=pred_world,x=fyear+1, color="World")) +
  geom_line(data=df_clean[df_clean$isin=="SG1S83002349" & df_clean$fyear < 2021,], aes(y=pred_world,x=fyear+1, color="World")) + 
  geom_point(data=df_preds, aes(y=preds, x=fyear, color=model), size=1.5, shape=18) +
  ylab('Revenue (real or predicted)') + xlab('Year')
ggplotly(plot)
```


## In Sample Accuracy {data-background="../Backgrounds/case.jpg" class="default"}

```{r, echo=F, include=F}
actual_series <- df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,]$revt_lead
uol_series <- uol[uol$fyear < 2021,]$pred_uol
base_series <- df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,]$pred_base
macro_series <- df_SG_macro[df_SG_macro$isin=="SG1S83002349" & df_SG_macro$fyear < 2021,]$pred_macro
world_series <- df_clean[df_clean$isin=="SG1S83002349" & df_clean$fyear < 2021,]$pred_world
```

```{r}
# series vectors calculated here -- See appendix
rmse <- function(v1, v2) {
  sqrt(mean((v1 - v2)^2, na.rm=T))
}

rmse <- c(rmse(actual_series, uol_series), rmse(actual_series, base_series),
          rmse(actual_series, macro_series), rmse(actual_series, world_series))
names(rmse) <- c("UOL 2018 UOL", "UOL 2018 Base", "UOL 2018 Macro", "UOL 2018 World")
rmse
```

> Why is UOL the best for *in sample*?

<div class="fragment exclude">

> UOL is trained to minimize variation only in that context.  It is potentially overfitted, meaning it won't predict well *out of sample*.  Out of sample prediction is much more useful than in sample, however.

</div>

## Actual Accuracy {data-background="../Backgrounds/case.jpg" class="default"}

> UOL posted a \$`r round(uol_fullyear %>% filter(fyear == 2022) %>% pull(revt)/1000, 1)`B in revenue in 2022.

```{r}
preds
```

> Why is the global model better?

- Consider UOL's business model
    - [2022 annual report](https://www.uol.com.sg/wp-content/uploads/uol-reports/files/UOL_Annual_Report_2022.pdf)



# End Matter {data-background-video="../Backgrounds/section.mp4"}

## Wrap up {data-background="../Backgrounds/ending.jpg"}

- For next week:
    - 2 chapters on Datacamp (optional)
    - First assignment
        - Turn in on eLearn before class in 2 weeks
        - You can work on this in *pairs* or *individually*
- Survey on the class session at this QR code:

{{< qrcode rmc.link/420survey2 qrsurvey width=200 height=200 >}}

## Packages used for these slides {data-background="../Backgrounds/code.jpg" visibility="uncounted"}

- `{broom}`
- `{DT}`
- `{downlit}`
- `{fixest}`
- `{kableExtra}`
- `{knitr}`
- `{plotly}`
- `{quarto}`
- `{revealjs}`
- `{tidyverse}`

## Custom code {data-background="../Backgrounds/code.jpg" class="small" visibility="uncounted"}

```{r, eval=FALSE}
# Graph showing squared error (slide 4.6)
uolg <- uol[,c("at","revt")]
uolg$resid <- mod1$residuals
uolg$xleft <- ifelse(uolg$resid < 0,uolg$at,uolg$at - uolg$resid)
uolg$xright <- ifelse(uolg$resid < 0,uolg$at - uolg$resid, uol$at)
uolg$ytop <- ifelse(uolg$resid < 0,uolg$revt - uolg$resid,uol$revt)
uolg$ybottom <- ifelse(uolg$resid < 0,uolg$revt, uolg$revt - uolg$resid)
uolg$point <- TRUE

uolg2 <- uolg
uolg2$point <- FALSE
uolg2$at <- ifelse(uolg$resid < 0,uolg2$xright,uolg2$xleft)
uolg2$revt <- ifelse(uolg$resid < 0,uolg2$ytop,uolg2$ybottom)

uolg <- rbind(uolg, uolg2)

uolg %>% ggplot(aes(y=revt, x=at, group=point)) + 
         geom_point(aes(shape=point)) + 
         scale_shape_manual(values=c(NA,18)) + 
         geom_smooth(method="lm", se=FALSE) +
         geom_errorbarh(aes(xmax=xright, xmin = xleft)) + 
         geom_errorbar(aes(ymax=ytop, ymin = ybottom)) + 
         theme(legend.position="none")
```

```{r, eval=FALSE}
# Chart of mean revt_lead for Singaporean firms (slide 12.6)
df_clean %>%                                           # Our data frame
  filter(fic=="SGP") %>%                               # Select only Singaporean firms
  group_by(isin) %>%                                   # Group by firm
  mutate(mean_revt_lead=mean(revt_lead, na.rm=T)) %>%  # Determine each firm's mean revenue (lead)
  slice(1) %>%                                         # Take only the first observation for each group
  ungroup() %>%                                        # Ungroup (we don't need groups any more)
  ggplot(aes(x=mean_revt_lead)) +                      # Initialize plot and select data
  geom_histogram(aes(y = ..density..)) +               # Plots the histogram as a density so that geom_density is visible
  geom_density(alpha=.4, fill="#FF6666")               # Plots smoothed density 
```

```{r, include=F, echo=F}
# Exports for the following week
save(df_clean, forecast2, forecast3, uol, forecast4, file = "../../Data/Session_2_export.RData")
```

