---
title: "ACCT 420: Ensembling and Ethics"
author: 
  - name: Dr. Richard M. Crowley
    email: rcrowley@smu.edu.sg
    affiliation: https://rmc.link/
title-slide-attributes:
  data-background-video: "../Backgrounds/title.mp4"
  data-background-video-loop: "true"
  data-background-size: stretch
format:
  revealjs:
    theme: default
    css: quarto_white.css
    navigation-mode: vertical
    controls: true
    transition: slide
    background-transition: fade
    transition-speed: default
    width: 1536
    height: 864
    logo: ../Logos/logo2.png
    code-link: true
    code-line-numbers: false
    chalkboard:
      theme: whiteboard
    auto-stretch: false
revealjs-plugins:
  - attribution
execute:
  echo: true
  warning: false
  message: false
editor: source
---

# Front Matter {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

<!--# Note that the downlit and xml2 pacakges are required to compile -->
<!--# Installed extensions include `attribution`, `qrcode`, and `material-icons` -->

```{r helpers, echo=FALSE, warning=FALSE, message=F}
library(knitr)
library(kableExtra)

#' Pretty formatted HTML tables
#' 
#' @param df The dataframe to format as a table
#' @param names Names to use for columns if not using the df column names. Vector
#' @param highlight_cols Columns to highlight. Default is c(1) for highlighting the first row. Use c() to remove.
#' @param highlight_rows Rows to highlight. Default is c() to not highlight any rows.
#' @param color_rows Rows to highlight in color. Default is c() to not highlight any rows.
#' @param highlight_end_rows Rows to highlight. Uses a reverse indexing where 1 is the last row, 2 is second to last, etc.
#' @param full Use to span the full width of a slide. FALSE by default.
#' @param fixed_header Whether the first row should be treated as a header row. TRUE by default.
html_df <- function(df, names=NULL, highlight_cols=c(1), highlight_rows=c(), color_rows=c(), highlight_end_rows=c(), full=F, fixed_header=T) {
  if(!length(names)) {
    names=colnames(df)
  }
  kbl(df,"html", col.names = names, align = c("l",rep('c',length(cols)-1))) %>%
    kable_paper(c("striped","hover"), full_width=full, fixed_thead=F, html_font = "\"Source Sans Pro\", Helvetica, sans-serif") %>%
    {if(length(highlight_cols)) column_spec(., highlight_cols,bold=T, background = "#ffffff99") else .} %>%
    {if(length(highlight_rows)) row_spec(., highlight_rows,bold=T, background = "#ffffff99") else .} %>%
    {if(length(color_rows)) row_spec(., color_rows,bold=T, background = "#aaaaff99") else .} %>%
    {if(length(highlight_end_rows)) row_spec(., nrow(df) + 1 - highlight_end_rows,bold=T, background = "#ffffff99") else .}
}

library(tidyverse)
```

<!--# List packages here that are not being correctly highlighted -->
<!--# The usual cause is calling the library from a echo=FALSE cell -->

```{r}
#| include: false
options(downlit.attached = c("knitr", "kableExtra", "tidyverse"))
```

## Learning objectives {data-background="../Backgrounds/default.jpg"}

:::: columns

::: {.column width="50%"}

![](../Course_Diagrams/Map-9.png){fig-alt="Roadmap for the course"}

:::

::: {.column width="50%"}
- <b>Theory:</b>
    - Ensembling
    - Ethics
- <b>Application:</b>
    - Fraud detection
    - Fairness
    - Data security
- <b>Methodology:</b>
    - Ensembling
    - SHAP
    - Critical thinking

:::

::::

## Final homework {data-background="../Backgrounds/default.jpg" class="default exclude"}

- Examining data from "The Examiner", a "SpamClickBait News Dataset"
    - For the sake of running things in a reasonable timeframe, I am providing you a random sample of 20,000 of the 170182 articles from 2015 (and ignoring the other 2,919,599 articles in the data set)
         - You are welcome to explore these on your own, however
    - Additionally examining a data set from Reddit's *r/worldnews* group
- Examining the "Social Credit System" in China

> Due by the final class (so I can review it briefly with you on the last day)

## Final exam {data-background="../Backgrounds/studying.jpg" class="default exclude"}

- Format:
    - ~25% Multiple choice related to coding
    - ~75% Long format: Proposing methods, interpreting results
- For studying
    - I have posted a full practice final exam
    - For text analytics, I have posted the Enron case, analyzing its fraud using a collection of Enron's internal emails
- The best way to study is to practice doing analytics
    - Your group projects are good example of this
    - Consider working out another problem on your own or with a group
        - Is there anything you ever wanted to know about businesses?
        - Feel free to schedule a consultation to go over your findings

# Ensembles {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

## What are ensembles? {data-background="../Backgrounds/reference.jpg"}

- Ensembles are *models made out of models*
- Ex.: You train 3 models using different techniques, and each seems to work well in certain cases and poorly in others
    - If you use the models in isolation, any of them would do an OK (but not great) job
    - If you make a model using all three, you can get better performance if their strengths all shine through
- Ensembles range from simple to complex
    - Simple: a (weighted) average of a few model's predictions

![](../Figures/ens_simple.png){height="150px" fig-align="center"}

## When are ensembles useful? {data-background="../Backgrounds/default.jpg"}

1. You have multiple models that are all decent, but none are great
    - And, ideally, the models' predictions are not highly correlated

![](../Figures/ens_corr.png){height="400px" fig-align="center"}    

## When are ensembles useful? {data-background="../Backgrounds/default.jpg"}

2. You have a really good model and a bunch of mediocre models
    - And, ideally the mediocre models are not highly correlated

![](../Figures/ens_great.png){height="400px" fig-align="center"}

## When are ensembles useful? {data-background="../Backgrounds/default.jpg"}

3. You really need to get just a bit more accuracy/less error out of the model, and you have some other models lying around
4. You want a more stable model
    - It helps to stabilize predictions by limiting the effect of errors or outliers produced by any one model on your prediction
    - Think: Diversification (like in finance)


## A simple ensemble (averaging) {data-background="../Backgrounds/default.jpg"}

- For continuous predictions, simple averaging is viable
    - Often you may want to weight the best model a bit higher
- For binary or categorical predictions, consider averaging *ranks*
    - i.e., instead of using a probability from a logit, use ranks 1, 2, 3, etc.
    - Ranks average a bit better, as scores on binary models (particularly when evaluated with measures like AUC) can have extremely different variances across models
        - In which case the ensemble is really just the most volatile model's prediction...
            - Not much of an ensemble

![](../Figures/ens_simple_rank.png){height="150px" fig-align="center"}

## A more complex ensemble (voting model) {data-background="../Backgrounds/default.jpg"}

- If you have a model the is very good at predicting a binary outcome, ensembling can still help
    - This is particularly true when you have other models that capture different aspects of the problem
- Let the other models vote against the best model, and use their prediction if they are above some threshold of agreement

![](../Figures/ens_voting.png){height="300px" fig-align="center"}

## A lot more complex ensemble {data-background="../Backgrounds/default.jpg"}

- Stacking models (2 layers)
    1. Train models on subsets (folds) of the training data
    2. Make predictions for each model on the folds it wasn't applied to
    2. Train a new model that takes those predictions as inputs (and optionally the dataset as well)
- Blending (similar to stacking)
    - Like stacking, but using predictions on a hold out sample instead of folds (and thus all models are using the same data for predictions)

![](../Figures/ens_stacking.png){height="250px" fig-align="center"}

## Simple ensemble example {data-background="../Backgrounds/coding.jpg"}

- Ensembling all our fraud detection models from Session 6

```{r, message=F, warnings=F, message=F, results=F}
df <- readRDS('../../Data/Session_6_models.rds')
df2 <- df %>% mutate(AAER = ifelse(AAER==0,0,1))
library(xgboost)

# Prep data
train_x <- model.matrix(AAER ~ ., data=df2[df2$Test==0,-1])[,-1]
train_y <- model.frame(AAER ~ ., data=df2[df2$Test==0,])[,"AAER"]
test_x <- model.matrix(AAER ~ ., data=df2[df2$Test==1,-1])[,-1]
test_y <- model.frame(AAER ~ ., data=df2[df2$Test==1,])[,"AAER"]

set.seed(468435)  #for reproducibility
xgbCV <- xgb.cv(max_depth=5, eta=0.10, gamma=5, min_child_weight = 4,
                subsample = 0.5, objective = "binary:logistic", data=train_x,
                label=train_y, nrounds=100, eval_metric="auc", nfold=10,
                stratified=TRUE, verbose=0)
fit_ens <- xgboost(params=xgbCV$params, data = train_x, label = train_y,
                   nrounds = which.max(xgbCV$evaluation_log$test_auc_mean),
                   verbose = 0)
```

> We trained an XGBoost model on all our prior model outputs

## Simple ensemble results {data-background="../Backgrounds/coding.jpg"}



```{r, echo=F, warning=F, message=F, fig.height=3.5}
library(yardstick)
all_x <- model.matrix(AAER ~ ., data=df[,-1])[,-1]
df$pred_ens <- predict(fit_ens, all_x, type="response")

df_test <- df %>% filter(Test==1)

aucs <- c(df_test %>% roc_auc(AAER, pred_ens, event_level='second') %>% pull(.estimate),
          df_test %>% roc_auc(AAER, pred_BCE, event_level='second') %>% pull(.estimate),
          df_test %>% roc_auc(AAER, pred_lmin, event_level='second') %>% pull(.estimate),
          df_test %>% roc_auc(AAER, pred_xgb, event_level='second') %>% pull(.estimate))
names(aucs) <- c("Ensemble", "Logit (BCE)", "Lasso (lambda.min)", "XGBoost")


curve_out_ens <- df_test %>% roc_curve(AAER, pred_ens, event_level='second')
curve_out_BCE <- df_test %>% roc_curve(AAER, pred_BCE, event_level='second')
curve_out_lmin <- df_test %>% roc_curve(AAER, pred_lmin, event_level='second')
curve_out_xgb <- df_test %>% roc_curve(AAER, pred_xgb, event_level='second')

ggplot() +
  geom_line(data=curve_out_BCE, aes(y=sensitivity, x=1-specificity, color="BCE")) + 
  geom_line(data=curve_out_lmin, aes(y=sensitivity, x=1-specificity, color="LASSO, lambda.min")) + 
  geom_line(data=curve_out_xgb, aes(y=sensitivity, x=1-specificity, color="XGBoost")) +
  geom_line(data=curve_out_ens, aes(y=sensitivity, x=1-specificity, color="Ensemble")) +
  geom_abline(slope=1)
```

```{r}
aucs  # Out of sample
```

> The ensemble has the best performance

## What drives the ensemble? {data-background="../Backgrounds/coding.jpg"}

```{r, fig.height=4.5}
xgb.train.data = xgb.DMatrix(train_x, label = train_y, missing = NA)
col_names = attr(xgb.train.data, ".Dimnames")[[2]]
imp = xgb.importance(col_names, fit_ens)
# Variable importance
xgb.plot.importance(imp)
```


## Practicalities {data-background="../Backgrounds/default.jpg"}

- Methods like stacking or blending are much more complex than a simple averaging or voting based ensemble
    - But in practice they perform slightly better

> Recall the tradeoff between complexity and accuracy!

- As such, we may not prefer the complex ensemble in practice, unless we only care about accuracy

> Example: In 2009, Netflix awarded a \$1M prize to the BellKor's Pragmatic Chaos team for beating Netflix's own user preference algorithm by >10%.  The alogorithm was so complex that Netflix <a href="https://www.wired.com/2012/04/netflix-prize-costs/">never used it</a>.  It instead used a simpler algorithm with an 8% improvement.

## [Geoff Hinton's] **Dark knowledge** {data-background="../Backgrounds/spooky.jpg"}

- Complex ensembles work well
- Complex ensembles are exceedingly computationally intensive
    - This is bad for running on small or constrained devices (like phones)

<div class="fragment">

> Dark knowledge

- We can (almost) always create a simple model that approximates the complex model
    - Interpret the above literally -- we can train a model to fit the model

</div>

## **Dark knowledge** {data-background="../Backgrounds/spooky.jpg"}

- Train the simple model not on the actual DV from the training data, but on the best algorithm's (softened) prediction for the training data
- Somewhat surprisingly, this new, simple algorithm can work almost as well as the full thing!

![](../Figures/ens_hinton.png){height="400px" fig-align="center"}

## An example of this **dark knowledge** {data-background="../Backgrounds/spooky.jpg"}

- Google's full model for interpretting human speech is >100GB
    - As of October 2019
- In Google's Pixel 4 phone, they have human speech interpretation running locally *on the phone*
    - Not in the cloud like it works on any other Android phone

> How did they do this?

- They can approximate the output of the complex speech model using a 0.5GB model
- 0.5GB isn't small, but it's small enough to run on a phone

## Learning more about Ensembling {data-background="../Backgrounds/default.jpg"}

- <a href="https://scikit-learn.org/stable/modules/ensemble.html">Scikit-learn's documentation on ensemble methods it supports</a>
- <a href="http://www.ttic.edu/dl/dark14.pdf">Geoff Hinton's Dark Knowledge slides</a>
    - For more details on *dark knowledge*, applications, and the softening transform
    - His interesting (though highly technical) <a href="https://www.reddit.com/r/MachineLearning/comments/2lmo0l/ama_geoffrey_hinton/">Reddit AMA</a>
- <a href="https://web.archive.org/web/20210924023432/https://mlwave.com/kaggle-ensembling-guide/">Kaggle Ensembling Guide</a>
    - A comprehensive list of ensembling methods with some code samples and applications discussed
- <a href="https://web.archive.org/web/20190305042959/https://blog.statsbot.co/ensemble-learning-d1dcd548e936?gi=e42c4e39a637">Ensemble Learning to Improve Machine Learning Results</a>
    - Nicely covers bagging and boosting (two other techniques)

> There are many ways to ensemble, and there is no specific guide as to what is best.  It may prove useful in the group project, however.

# Ethics: Fairness {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

## In class reading with case {data-background="../Backgrounds/group.jpg"}

- From Datarobot's Colin Preist:
    - <a href="https://blog.datarobot.com/four-keys-to-avoiding-bias-in-ai">Four Keys to Avoiding Bias in AI</a>
        - Short link: <a href="https://blog.datarobot.com/four-keys-to-avoiding-bias-in-ai">rmc.link/420class9</a>
    - *You only need to read the Case portion*
- The four points:
    1. Don't trust black boxes
    2. Check for direct discrimination by the model via inputs
    3. Check for indirect discrimination
    4. Use training data representative of the outcome you want

> What was the issue in the case?  Where might similar issues crop up in business?  Any examples?

## Brainstorming... {data-background="../Backgrounds/rain.jpg" class="default present-only"}

<iframe allowfullscreen frameborder="0" height="600px" mozallowfullscreen src="https://app.wooclap.com/events/RFQIRZ/questions/5f8e91c25cc8025609a96bc8" width="100%"></iframe>

## An example of unexpected algorithm behavior {data-background="../Backgrounds/case.jpg" class="default present-only"}

Microsoft's Tay AI chatbot in 2016:

![](../Figures/tay_gerveis.png){width="450px" fig-align="center"}

::: {.fragment}

![](../Figures/tay_after.png){width="450px" fig-align="center"}

:::

## Examples of Reputational damage {data-background="../Backgrounds/case.jpg"}

- [Microsoft's Tay](https://www.theguardian.com/technology/2016/mar/26/microsoft-deeply-sorry-for-offensive-tweets-by-ai-chatbot) and [their response](https://blogs.microsoft.com/blog/2016/03/25/learning-tays-introduction/)
- [Coca-Cola: Go make it happy](http://gawker.com/make-hitler-happy-the-beginning-of-mein-kampf-as-told-1683573587)
- [Google: Google Photos mistakenly labels black people 'gorillas'](https://bits.blogs.nytimes.com/2015/07/01/google-photos-mistakenly-labels-black-people-gorillas/)
- [Machine Bias](https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing)
    - ProPublica's in depth look at racial bias in US courts' risk assessment algorithms (as of May 2016)
    - Note that the number of true positives divided by the number of all positives is [more or less equal across ethnicities](https://www.technologyreview.com/s/607955/inspecting-algorithms-for-bias/)

![](../Figures/court_bias_US.png){height="230px" fig-align="center"}

##  {data-background="../Backgrounds/default.jpg"}

[![](../Figures/ML_care.png){height="750px" fig-align="center"}](https://newgeneralization.github.io/slides/PercyLiang.pdf)

## Fairness is complex! {data-background="../Backgrounds/default.jpg"}

- There are many different (and disparate) definitions of fairness
    - Arvind Narayanan's [Tutorial: 21 fairness definitions and their politics](https://www.youtube.com/watch?v=jIXIuYdnyyk)
- For instance, in the court system example:
    - If an algorithm has the same accuracy across groups, but rates are different across groups, then true positive and false positive rates must be different!

> Fairness requires considering different perspectives and identifying which perspectives are most important from an ethical perspective

## How could the previous examples be avoided? {data-background="../Backgrounds/default.jpg"}

- Filtering data used for learning the algorithms
    - Microsoft Tay should have been more careful about the language used in retraining the algorithm over time
        - Particularly given that the AI was trained on public information on Twitter, where coordination against it would be simple
- Filtering output of the algorithms
    - Coca Cola could check the text for content that is likely racist, classist, sexist, etc.
- Google may have been able to avoid this using training dataset that was sensitive to potential problems
    - For instance, using a *balanced* data set across races
    - As an intermediary measure, they removed searching for gorillas and its associated label from the app

## Examining ethics for algorithms {data-background="../Backgrounds/default.jpg"}

1. Understanding the problem and its impact
    - Think about the effects the algorithm will have!
        - Will it drastically affect lives?  If yes, exercise more care!
    - Think about what you might expect to go wrong
        - What biases might you expect or might find in the data?
        - What biases do people doing the same task exhibit?
2. Manual inspection
    - Check model outputs against problematic indicators
    - Test the algorithm before putting it into production
3. Use methods like <a href="https://github.com/slundberg/shap">SHAP</a> to explain models
4. Some thoughts on the matter from leading companies:
    - Google: [Responsible AI practices](https://ai.google/responsibility/responsible-ai-practices/)
    - Meta: [Meta’s Civil Rights Progress](https://about.fb.com/news/2023/03/an-update-on-metas-civil-rights-progress/)
    - OpenAI: [How should AI systems behave, and who should decide?](https://openai.com/blog/how-should-ai-systems-behave)

# Examining Fairness with SHAP {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

## What is SHAP? {data-background="../Backgrounds/default.jpg"}

> SHAP aims to provide an explanation of the importance of model inputs in explaining model output

- SHAP: **SH**apley **A**dditive ex**P**lanations
- SHAP is...
    - game theoretic
    - theory driven
    - a model itself: a model that explains models
- SHAP provides a simple to understand output

::: aside

SHAP is based on Shapley (1953), "A value for n-person games." SHAP is derived in Lundberg and Lee (2017)

:::

## Key points of SHAP {data-background="../Backgrounds/default.jpg"}

1. SHAP maintains *local accuracy*
    - It can accurately model outputs on a small subset of data
2. SHAP maintains *missingness*
    - It only uses data used by the model
3. SHAP maintains *consistency*
    - The final output of SHAP follows the same rules as Utility in economics

> SHAP will give us something like a marginal effect, but for *any* model, including complex ML models

## Data and approach {data-background="../Backgrounds/default.jpg"}

- City of Chicago salaries
    - 33,586 employees
- Trained using a simple XGBoost model
- Features:
    - Job title
    - Full time or part time
    - Salaried or hourly
    - Department
    - Gender (based on name)
- Code to replicate is on eLearn

```{r, include=F, echo=F, message=F, warning=F}
library(gender)
df <- read_csv('../../Data/Session_9-City_of_Chicago_Salary_2019.04.csv')
names(df) <- make.names(names(df),unique = TRUE)
df <- df %>%
  mutate(Full.Time = ifelse(Full.or.Part.Time == 'F',1 ,0),
         Salaried = ifelse(Salary.or.Hourly == 'Salary', 1, 0),
         Salary = ifelse(is.na(Annual.Salary), Typical.Hours * Hourly.Rate, Annual.Salary),
         Obs = n()) %>%
  group_by(Job.Titles) %>%
  mutate(Job.Titles.Freq = n()/Obs) %>%
  ungroup() %>%
  group_by(Department) %>%
  mutate(Department.Freq = n()/Obs) %>%
  ungroup() %>%
  mutate(Job.Titles = ifelse(Job.Titles.Freq > 0.03, Job.Titles, 'Other'),
         Department = ifelse(Department.Freq > 0.05, Department, 'Other'),
         Job.Titles = factor(Job.Titles),
         Department = factor(Department),
         first_name = str_extract(df$Name, '(?<=,[:space:]{2})[:alpha:]+'))

genders <- gender(df$first_name) %>% distinct(name, .keep_all = T)
genders <- genders %>%
  mutate(Female = ifelse(proportion_female > proportion_male,1,0))
df <- left_join(df, genders[, c('name', 'Female')], by = c("first_name" = "name"))

train_x <- model.matrix(Salary ~ Job.Titles + Department + Full.Time + Salaried + Female, data=df)[,-1]
train_y <- model.frame(Salary ~ Job.Titles + Department + Full.Time + Salaried + Female, data=df)[, "Salary"]

set.seed(654687)  #for reproducibility
xgbCV <- xgb.cv(max_depth=6, eta=0.30, gamma=0, min_child_weight = 1,
                subsample = 1, booster = 'gbtree', data=train_x,
                label=train_y, nrounds=100, eval_metric="rmse", nfold=10,
                verbose=0)

fit_xgb <- xgboost(params=xgbCV$params, data = train_x, label = train_y,
                   nrounds = which.min(xgbCV$evaluation_log$test_rmse_mean))
```

## SHAP values {data-background="../Backgrounds/case_digital.jpg"}

```{r, warning=F}
library('SHAPforxgboost')
#https://liuyanguu.github.io/post/2019/07/18/visualization-of-shap-for-xgboost/



vals <- shap.values(xgb_model = fit_xgb, X_train = train_x)
vals$mean_shap_score
```

## SHAP values by gender {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}

**Female**

```{r}
#| eval: true
#| echo: true

vals$shap_score[train_x[, 'Female'] == 1] %>%
  colMeans()

```

:::

::: {.column width="50%"}

**Male**

```{r}
#| eval: true
#| echo: true

vals$shap_score[train_x[, 'Female'] == 0] %>%
  colMeans()
```

:::

::::

> What do these differences mean?

## Visualizing SHAP across the population {data-background="../Backgrounds/case_digital.jpg"}

```{r, fig.height = 4.5, fig.width = 8, message=F}
shap_long <- shap.prep(xgb_model = fit_xgb, X_train = train_x)
shap.plot.summary(shap_long)
```

## Visualizing differences in gender within a job {data-background="../Backgrounds/default.jpg"}

```{r}
#| eval: true
#| echo: true
shap.plot.dependence(data_long = shap_long, x = 'Female', y = 'Female',
                           color_feature = 'Job.TitlesPOLICE OFFICER', jitter_width = 0.05)

```

## SHAP force plot: natural groupings in the data {data-background="../Backgrounds/case_digital.jpg"}


```{r, fig.height=4.5, fig.width=6, message=F}
plot_data <- shap.prep.stack.data(shap_contrib = vals$shap_score,
                                  top_n = 5, n_groups = 6)
shap.plot.force_plot_bygroup(plot_data)
```

## Areas where ethics is particularly important {data-background="../Backgrounds/default.jpg"}

- Anything that impacts people's livelihoods
    - Legal systems
    - Healthcare and insurance systems
    - Hiring and HR systems
    - Finance systems like credit scoring
    - Education
- Anything where failure is catastrophic
    - Voting systems
    - Engineering or transportation systems
        - Such as the <a href="https://www.straitstimes.com/singapore/joo-koon-collision-software-glitch-in-signalling-system-results-in-second-train-misjudging">Joo Koon MRT Collision in 2017</a>
        - <a href="http://moralmachine.mit.edu/">Self driving cars</a> (<a href="https://venturebeat.com/2018/10/24/mit-study-explores-the-trolley-problem-and-self-driving-cars/">Results summary</a>)

> Article: <a href="https://www.wired.com/2014/11/algorithms-great-can-also-ruin-lives/">Algorithms are great and all, but they can also ruin lives

## Research on fairness {data-background="../Backgrounds/neural-net.jpg"}

- Excerpt below from [Universal Sentence Encoder](https://arxiv.org/abs/1803.11175)

![](../Figures/USE_fairness.png){height="320px" fig-align="center"}

> Compares a variety of unintended associations (top) and intended associations (bottom) across Global Vectors (GloVe) and USE

## Ethical implications: Case {data-background="../Backgrounds/group.jpg"}

- In Chicago, IL, USA, they are using a system to rank arrested individuals, and they use that rank for *proactive* policing
- Read about the system here: [rmc.link/420class9-2](https://rmc.link/420class9-2)

> What risks does such a system pose?

> How would you feel if a similar system was implemented in Singapore?

## Brainstorming... {data-background="../Backgrounds/rain.jpg" class="default present-only"}

<iframe allowfullscreen frameborder="0" height="600px" mozallowfullscreen src="https://app.wooclap.com/events/RFQIRZ/questions/5f8e91fd5271ff4fa2cfe360" width="100%"></iframe>

## Brainstorming... {data-background="../Backgrounds/rain.jpg" class="default present-only"}

<iframe allowfullscreen frameborder="0" height="600px" mozallowfullscreen src="https://app.wooclap.com/events/RFQIRZ/questions/5f8e928eb553c711909075e7" width="100%"></iframe>

## Other references {data-background="../Backgrounds/default.jpg"}

> [Kate Crawford's NIPS 2017 Keynote: "The Trouble with Bias" (video)](https://www.youtube.com/watch?v=fMym_BKWQzk)

<iframe width="560" height="315" src="https://www.youtube.com/embed/fMym_BKWQzk?si=LhII8GIlevi8_Zkr" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

# Ethics: Data security {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

## A motivating example {data-background="../Backgrounds/default.jpg" class="default present-only"}

[Withheld from all public copies]

## Recall the Walmart data {data-background="../Backgrounds/case.jpg" class="default present-only"}

- 45 stores from different regions
    - Each stores' revenue by department
    - **Each store is anonymized with an ID**
        - A number between 1 and 45
- Other data: CPI, labor, weather, gas price

## Recall the Walmart data {data-background="../Backgrounds/question.jpg" class="default present-only"}

> What if I said I could tell you which store ID 32 and 11 were located?

<div class="fragment">
- Walmart would likely not be happy about this
    - Competitors could use this to extract private information and strategize against Walmart
- But...

<div class="fragment">
- They did a bad job anonymizing the data, making this possible
</div>
</div>

## Where are stores 32 and 11? {data-background="../Backgrounds/case_digital.jpg" class="default present-only"}

```{r, eval=F, include=F}
# Code to de-anonymize Walmart
# Private, do not release publicly
# Code saves results to an RDS so there is no cross-over between this code and the generated code file
library(stringr)
library(magrittr)
library(lubridate)
library(leaflet)
wmt <- read_csv('../../Data/Session_4/Wallmart-store-data/WMT_features.csv')
# file: air_temp.YYYY
# lon, lat, jan, feb, mar, ..., dec

year1 = 2010
year2 = 2013
for (year in year1:year2) {
  temp <- read_fwf('../../Data/air_temp.2010',
            fwf_widths(c(9,8,8,8,8,8,8,8,8,8,8,8,8,8),
              col_names=c("lon","lat",paste0(year, str_pad(1:12,2,pad="0")))))
  if (year == year1) {
    temps = temp
  } else {
    temps = temps %>% full_join(temp)
  }
}
# Temperatures by station lat lon
tdf <- temps %>% gather(month, temp, `201001`:`201312`)

# Restrict to US area stations
tdf %<>% filter(-124.7844079 <= lon, lon <= -66.9513812,
                24.7433195 <= lat, lat <= 49.3457868)

# generate monthly temperatures for Walmart stores
# create an adjusted temp that can simply be averaged within month
wmt %<>%
  mutate(month = month(Date),
         year = year(Date)) %>%
  arrange(Store, Date) %>%
  group_by(Store) %>%
  mutate(lag_temp = lag(Temperature)) %>%
  ungroup() %>%
  group_by(Store, year, month) %>%
  mutate(days_left = days_in_month(Date) - day(Date),
         temp2 = ifelse(row_number() == 1, Temperature,
                 ifelse(row_number() == n(), (7 * Temperature +  days_left* lag_temp)/(7 + days_left),
                        Temperature)),
         weigh = ifelse(row_number() == 1, day(Date),
                 ifelse(row_number() == n(), 7 + days_left,
                        7)),
         temp_mo = weighted.mean(temp2, weigh)) %>%
  slice(1) %>%
  ungroup() %>%
  mutate(temp_mo = (temp_mo - 32) * 5/9,
         month = paste0(year(Date), str_pad(month(Date),2,pad="0"))) %>%
  select(Store, month, temp_mo)

rmse <- function(v1, v2) {
  sqrt(mean((v1 - v2)^2, na.rm=T))
}

map_store <- function(store_df, tdf, store) {
  df <- store_df %>% filter(Store==store)
  tdf %<>% left_join(df)
  tdf %>%
    group_by(lat, lon) %>%
    mutate(dist = rmse(temp, temp_mo)) %>%
    slice(1) %>%
    ungroup() %>%
    arrange(dist)
}

map_gen <- function(df) {
  pal <- colorNumeric(
    palette = colorRampPalette(c('red', 'blue'))(length(df$dist)), 
  domain = log(log(df$dist)))
  #binpal <- colorNumeric("Blues", domain=df$dist)
  #setView(lng = 136, lat = 34, zoom=4) %>%
  leaflet() %>%
  addTiles() %>% 
  addCircleMarkers(data=df, lng=~lon, lat=~lat, radius=3, color = ~pal(log(log(dist))))
}

get_min_error <- function(wmt, tdf) {
  mins = data.frame(Store=c(), dist=c(), lat=c(), lon=c())
    for (store in 1:45) {
      new <- map_store(wmt, tdf, store) %>%
        slice(1) %>%
        select(Store, dist, lat, lon)
      mins %<>%
        bind_rows(new)
    }
  mins$Store <- 1:45
  mins
}

d <- get_min_error(wmt, tdf)
d %>% arrange(dist)
saveRDS(map_gen(d), '../../Data/Session_9_wmt_1.rds')
ds <- map_store(wmt, tdf, 33)
saveRDS(map_gen(ds), '../../Data/Session_9_wmt_2.rds')
#map_gen(ds[1:100,])
saveRDS(map_gen(ds[1:20,]), '../../Data/Session_9_wmt_3.rds')
ds <- map_store(wmt, tdf, 23)
#map_gen(ds[1:100,])
saveRDS(map_gen(ds[1:20,]), '../../Data/Session_9_wmt_4.rds')
ds <- map_store(wmt, tdf, 32)
#map_gen(ds[1:100,])
saveRDS(map_gen(ds[1:20,]), '../../Data/Session_9_wmt_5.rds')
ds <- map_store(wmt, tdf, 11)
#map_gen(ds[1:100,])
saveRDS(map_gen(ds[1:20,]), '../../Data/Session_9_wmt_6.rds')
```

```{r, echo=F}
g1 <- readRDS('../../Data/Session_9_wmt_1.rds')
g2 <- readRDS('../../Data/Session_9_wmt_2.rds')
g3 <- readRDS('../../Data/Session_9_wmt_3.rds')
g4 <- readRDS('../../Data/Session_9_wmt_4.rds')
g5 <- readRDS('../../Data/Session_9_wmt_5.rds')
g6 <- readRDS('../../Data/Session_9_wmt_6.rds')
```

:::: {.columns}

::: {.column width="50%"}
```{r echo=F, fig.height=5, fig.width=6}
g5
```
:::

::: {.column width="50%"}
```{r echo=F, fig.height=5, fig.width=6}
g6
```
:::

::::


## Where are stores 33 and 23? {data-background="../Backgrounds/case_digital.jpg" class="default present-only"}

:::: {.columns}

::: {.column width="50%"}
```{r echo=F, fig.height=5, fig.width=6}
g3
```
:::

::: {.column width="50%"}
```{r echo=F, fig.height=5, fig.width=6}
g4
```
:::

::::

## How did I back this out? {data-background="../Backgrounds/case_digital.jpg" class="default present-only"}

:::: {.columns}

::: {.column width="50%"}

- Walmart provides data on macro factors in each store's region
    - They don't give much detail on how these are calculated though
- Temperature
    - Unclear if this is average or high temperature
    - Provided weekly
- Cross reference with monthly temperature data from >1000 weather stations in the US
    - Check error (RMSE) for every station vs every store

:::

::: {.column width="50%"}

Heatmap of Store 33 RMSE
```{r echo=F, fig.height=5, fig.width=6}
g2
```

:::

::::

## Most likely location for all stores {data-background="../Backgrounds/case_digital.jpg" class="default present-only"}

```{r echo=F, fig.height=6, fig.width=10}
g1
```


## Anomymized data {data-background="../Backgrounds/default.jpg"}

- Generally we anonymize data because, while the data itself is broadly useful, providing full information could harm others or oneself
- Ex.: Studying drug usage can create a list of people with potentially uncaught criminal offenses
    - If one retains a list of identities, then there is an ethical dilemma:
        - Protect study participants by withholding the list
        - Provide the list to the government
            - This harms future knowledge generation by sowing distrust
    - Solution: Anonymous by design
- Website or app user behavior data
    - E.g.: FiveThirtyEight's [Uber rides dataset](https://www.kaggle.com/fivethirtyeight/uber-pickups-in-new-york-city)

> What could go wrong if the Uber data wasn't anonymized?

## Anonymization is tricky {data-background="../Backgrounds/default.jpg"}

> Both Allman & Paxson, and Partridge warn against relying on the anonymisation of data since *deanonymisation techniques are often surprisingly powerful*. Robust anonymisation of data is difficult, particularly when it has high dimensionality, as the anonymisation is likely to lead to an unacceptable level of data loss [3]. -- [TPHCB 2017](https://www.cl.cam.ac.uk/~drt24/papers/2017-ethical-issues.pdf)

- There are natural limits to anonymization, particularly when there is a limited amount of potential participants in the data
    - Example: Web browser tracking at [Panopticlick](https://panopticlick.eff.org/)

## Panopticklick {data-background="../Backgrounds/default.jpg" class="default present-only"}

![](../Figures/panop.png){width="800px" fig-align="center"}

## Responsibilities generating data {data-background="../Backgrounds/default.jpg"}

- Keep users as unidentifiable as feasible
- If you need to record people's private information, **make sure they know**
    - This is called *informed consent*
- If you are recording sensitive information, consider not keeping identities at all
    - Create a new, unique identifier (if needed)
    - Maintain as little identifying information as necessary
    - Consider using encryption if sensitive data is retained
    - Can unintentionally lead to infringements of *human rights* if the data is used in unintended ways

## Informed consent {data-background="../Backgrounds/reference.jpg"}

- When working with data about *people*, they should be informed of this and consent to the research, unless the data is publicly available
- From SMU's IRB Handbook: (2017 SEP 18 version)

<br><br>
<div style="font-size:20px">

- "*Informed consent*: Respect for persons requires that participants, to the degree
that they are capable, be given the opportunity to make their own judgments
and choices. When researchers seek participants’ participation in research
studies, they provide them the opportunity to make their own decisions to
participate or not by ensuring that the following adequate standards for
informed consent are satisfied:
    - *Information*: Participants are given sufficient information about the
research study, e.g., research purpose, study procedures, risks,
benefits, confidentiality of participants’ data.
    - *Comprehension*: The manner and context in which information is
conveyed allows sufficient comprehension. The information is
organized for easy reading and the language is easily comprehended
by the participants.
    - *Voluntariness*: The manner in which researchers seek informed
consent from the participants to participate in the research study
must be free from any undue influence or coercion. Under such
circumstances, participants are aware that they are not obliged to
participate in the research study and their participation is on a
voluntary basis."
</div>

> Also, note the existence of the [PDPA law](https://www.pdpc.gov.sg/Overview-of-PDPA/The-Legislation/Personal-Data-Protection-Act) in Singapore

## Human rights {data-background="../Backgrounds/reference.jpg"}

- Recall the drug users example
    - If data was collected without their consent, and if it was not anonymized perfectly, then this could lead to leaking of drug user's information to others

> What risks does this pose?  Consider contexts outside Singapore as well.

## Responsibilities using data {data-background="../Backgrounds/default.jpg"}

> "The collection, or use, of a dataset of illicit origin to support research can be advantageous. For example, legitimate access
to data may not be possible, or the reuse of data of illicit origin is likely to require fewer resources than collecting data
again from scratch. In addition, the sharing and reuse of existing datasets aids reproducibility, an important scientific goal. The disadvantage is that ethical and legal questions may arise as a result of the use of such data" (<a href="https://www.cl.cam.ac.uk/~drt24/papers/2017-ethical-issues.pdf">source</a>)

## Responsibilities using data {data-background="../Backgrounds/default.jpg"}

- Respect for persons
    - Individuals should be treated as *autonomous agents*
        - People **are** people
    - Those without autonomy should be protected
- Beneficence
    1. Do not harm (ideally)
    2. Maximize possible benefits and minimize possible harms
        - This can be a natural source of conflict
- Justice
    - Benefits and risks should flow to the same groups -- don't use unwilling or disadvantaged groups who won't receive any benefit
        - [Extreme] example: <a href="https://www.cdc.gov/tuskegee/timeline.htm">Tuskegee Syphilis study</a>

> Experiments: [The Belmont Report](https://www.hhs.gov/ohrp/sites/default/files/the-belmont-report-508c_FINAL.pdf); Electronic data: [The Menlo Report](https://www.dhs.gov/sites/default/files/publications/CSD-MenloPrinciplesCORE-20120803_1.pdf)

## A final example {data-background="../Backgrounds/default.jpg" class="default present-only"}

[Withheld from all public copies]

## A final example {data-background="../Backgrounds/case.jpg" class="default present-only"}

![](../Figures-local_only/abel_noser_email.png){height="750px" fig-align="center"}

# End Matter {data-background-video="../Backgrounds/section.mp4"}

## Wrap up {data-background="../Backgrounds/ending.jpg"}

Today, we:

- Learned about combining models to create an even better model
    - And the limits to this as pointed out by Geoff Hinton
- Discussed the potential ethical issues surrounding:
    - AI algorithms
    - Data creation
    - Data usage
- Learned how to implement a tool for analyzing fairness: SHAP

- Survey on the class session at this QR code:

{{< qrcode https://rmc.link/420survey9 qrsurvey width=200 height=200 >}}

## For next week {data-background="../Backgrounds/ending.jpg"}

- For the next 2 weeks:
    - We will talk about neural networks and vector methods (which are generally neural network based)
        - These are important tools underpinning a lot of recent advancements
        - We will take a look at some of the advancements, and the tools that underpin them
    - If you would like to be well prepared, there is <a href="https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471">a nice introductory article here</a> (8 parts though)
         - Part 1 is good enough for next week, but part 2 is also useful
         - For those very interested in machine learning, parts 3 through 8 are also great, but more technical and targeted at specific applications like facial recognition and machine translation
    - Keep working on the group project

## Fun machine learning examples {data-background="../Backgrounds/ending.jpg"}

- Interactive:
    - <a href="https://research.google.com/semantris/">Semantris</a>
        - A game based on the Universal Sentence Encoder
    - <a href="https://magenta.tensorflow.org/sketch-rnn-demo">Draw together with a neural network</a>
        - click the images to try it out yourself!
    - <a href="https://quickdraw.withgoogle.com/">Google's Quickdraw</a>
    - <a href="https://teachablemachine.withgoogle.com/">Google's Teachable Machine</a>
    - <a href="https://distill.pub/2016/handwriting/">Four experiments in handwriting with a neural network</a>
- Non-interactive
    - <a href="https://blog.insightdatascience.com/hero2vec-d42d6838c941">Predicting e-sports winners with Machine Learning</a>

## Packages used for these slides {data-background="../Backgrounds/code.jpg" visibility="uncounted"}


- `{DT}`
- `{downlit}`
- `{gender}`
- `{kableExtra}`
- `{knitr}`
- `{leaflet}`
- `{plotly}`
- `{quarto}`
- `{revealjs}`
- `{SHAPforxgboost}`
- `{tidyr}`
- `{tidyverse}`
- `{yardstick}`

## NOTES {class="never"}


- Move Enron mention to next class <- 2020
    - How though?  Given the link to the final exam files <- 2021
    - 2022: make some decision
- Ensembles is still tough -- maybe define exactly what an ensemble is up front?
    - Even though it isn't really well defined.


