---
title: "ACCT 420: ML/AI for numbers and text"
author: 
  - name: Dr. Richard M. Crowley
    email: rcrowley@smu.edu.sg
    affiliation: https://rmc.link/
title-slide-attributes:
  data-background-video: "../Backgrounds/title.mp4"
  data-background-video-loop: "true"
  data-background-size: stretch
format:
  revealjs:
    theme: default
    css: quarto_white.css
    navigation-mode: vertical
    controls: true
    transition: slide
    background-transition: fade
    transition-speed: default
    width: 1536
    height: 864
    logo: ../Logos/logo2.png
    code-link: true
    code-line-numbers: false
    chalkboard:
      theme: whiteboard
    auto-stretch: false
revealjs-plugins:
  - attribution
execute:
  echo: true
  warning: false
  message: false
editor: source
---

# Front Matter {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

<!--# Note that the downlit and xml2 pacakges are required to compile -->
<!--# Installed extensions include `attribution`, `qrcode`, and `material-icons` -->

```{r helpers, echo=FALSE, warning=FALSE, message=F}
library(knitr)
library(kableExtra)

#' Pretty formatted HTML tables
#' 
#' @param df The dataframe to format as a table
#' @param names Names to use for columns if not using the df column names. Vector
#' @param highlight_cols Columns to highlight. Default is c(1) for highlighting the first row. Use c() to remove.
#' @param highlight_rows Rows to highlight. Default is c() to not highlight any rows.
#' @param color_rows Rows to highlight in color. Default is c() to not highlight any rows.
#' @param highlight_end_rows Rows to highlight. Uses a reverse indexing where 1 is the last row, 2 is second to last, etc.
#' @param full Use to span the full width of a slide. FALSE by default.
#' @param fixed_header Whether the first row should be treated as a header row. TRUE by default.
html_df <- function(df, names=NULL, highlight_cols=c(1), highlight_rows=c(), color_rows=c(), highlight_end_rows=c(), full=F, fixed_header=T) {
  if(!length(names)) {
    names=colnames(df)
  }
  kbl(df,"html", col.names = names, align = c("l",rep('c',length(cols)-1))) %>%
    kable_paper(c("striped","hover"), full_width=full, fixed_thead=F, html_font = "\"Source Sans Pro\", Helvetica, sans-serif") %>%
    {if(length(highlight_cols)) column_spec(., highlight_cols,bold=T, background = "#ffffff99") else .} %>%
    {if(length(highlight_rows)) row_spec(., highlight_rows,bold=T, background = "#ffffff99") else .} %>%
    {if(length(color_rows)) row_spec(., color_rows,bold=T, background = "#aaaaff99") else .} %>%
    {if(length(highlight_end_rows)) row_spec(., nrow(df) + 1 - highlight_end_rows,bold=T, background = "#ffffff99") else .}
}

library(tidyverse)
```

<!--# List packages here that are not being correctly highlighted -->
<!--# The usual cause is calling the library from a echo=FALSE cell -->

```{r}
#| include: false
options(downlit.attached = c("knitr", "kableExtra", "tidyverse"))
```

## Learning objectives {data-background="../Backgrounds/default.jpg"}

:::: columns

::: {.column width="50%"}

![](../Course_Diagrams/Map-9.png){fig-alt="Roadmap for the course"}

:::

::: {.column width="50%"}

- **Theory:**
    - Neural Networks (broad overview)
    - Vector space methods
- **Application:**
    - Neural networks for understanding textual data
    - Annual report sentiment
    - Building our own *mini ChatGPT*
- **Methodology:**
    - Vector methods
    - Neural networks

:::

::::

# Languages for ML/AI {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

## R for ML/AI {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}
**Older methods**

- `{caret}`
- `{randomForest}`
- `{nnet}`
- `{e1071}`
:::

::: {.column width="50%"}
**Best-in-class**

- `{glmnet}`: LASSO and elastic nets
- `{xgboost}`: XGBoost
- `{Prophet}`: ML for time series forecasting
- `{keras}`: Plugs into python's Keras
- `{H2O4GPU}`: Plugs into python's H2O
- `{spacyr}`: Plugs into python's SpaCy
- `{mlverse/torch}`: Plugs in to Torch
:::

::::

## Python for ML/AI {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}
**Older methods**

- Sci-kit learn -- one stop shop for most older libraries
- RPy2
- scipy + numpy + pandas + statsmodels
    - Add [Theano](http://deeplearning.net/software/theano/) in for GPU compute
:::

::: {.column width="50%"}
**Best-in-class**

- [*TensorFlow*](https://www.tensorflow.org/) (Google) -- Can do everything, but often cumbersome
- [pytorch](https://pytorch.org/)<a href=""></a> -- python specific Torch port that is currently very popular
- [gensim](https://radimrehurek.com/gensim/)<a href=""></a> -- "Topic modelling for humans"
- [H2O](https://www.h2o.ai/)<a href=""></a> (H2O)
- [caffe](http://caffe.berkeleyvision.org/)<a href=""></a> (Berkley)
- [caffe2](https://caffe2.ai/)<a href=""></a> (Facebook)
- [SpaCy](https://spacy.io/) -- Fast NLP processing
- [CoreNLP](https://stanfordnlp.github.io/CoreNLP/other-languages.html) -- through various wrappers to the Java library
:::

::::


## Others for ML/AI {data-background="../Backgrounds/default.jpg"}

- C/C++: Also a first class language for TensorFlow!
    - Really fast -- precompiled
    - Much more difficult to code in
- Swift: Strong TensorFlow support
- Javascript: Improving support from TensorFlow and others

## Why focus on TensorFlow? {data-background="../Backgrounds/default.jpg" .small}

:::: {.columns}

::: {.column width="50%"}

- It can run almost ANY ML/AI/NN algorithm
- It has good community support:
    - [TensorFlow Hub](https://tfhub.dev/) -- Premade algorithms for text, image, and video
    - [tensorflow/models](https://github.com/tensorflow/models) -- Premade code examples
        - The [research](https://github.com/tensorflow/models/tree/master/research) folder contains an amazing set of resources
    - [trax](https://github.com/google/trax) -- AI research models from Google Brain

:::

::: {.column width="50%"}

[![](../Figures/tf_light.png){width="300px" fig-align="center"}](https://www.tensorflow.org/lite/)
[![](../Figures/tf_magenta.png){width="300px" fig-align="center"}](https://magenta.tensorflow.org/)
[![](../Figures/tf_hub.png){width="300px" fig-align="center"}](https://tfhub.dev/)

:::

::::

> Note: Google appears to be sunsetting this, but many *older* algorithms are based on it.  They are shifting toward [PyTorch](https://pytorch.org/) and [JAX](https://github.com/google/jax)

## About PyTorch {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}

- Based on [Torch](http://torch.ch/) (for Lua)
- All the companies on the right are on the governing board
- Underpins models by [fast.ai](linkhttps://www.fast.ai/), [ELF](https://github.com/pytorch/elf), and [AllenNLP](linkhttps://allennlp.org/)
- Easier to use than Tensorflow
- Most *new* off-the-shelf models use it
- [Pytorch Hub](https://pytorch.org/hub/)
    - Has a variety of pretrained models available
    - A bit easier to work with than TensorFlow Hub

:::

::: {.column width="50%"}

![](../Figures/Pytorch_gov_board.png){width="600px" fig-align="center"}

:::

::::

## Other notable frameworks {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}

- [Caffe](http://caffe.berkeleyvision.org/)
    - Python, C/C++, Matlab
    - Good for image processing
- [Caffe2](https://caffe2.ai/)
    - C++ and Python
    - Still largely image oriented
- [Microsoft Cognitive Toolkit](https://www.microsoft.com/en-us/cognitive-toolkit/)
    - Python, C++
    - Scales well, good for NLP
- [H2O](https://www.h2o.ai/)
    - Python based
    - Integration with R, Scala...

:::

::: {.column width="50%"}

[![](../Figures/caffe2.svg){width="250px" fig-align="center"}](https://caffe2.ai/)
[![](../Figures/cntk.png){width="250px" fig-align="center"}](https://www.microsoft.com/en-us/cognitive-toolkit/)
[![](../Figures/h2o_logo.svg){width="250px" fig-align="center"}](https://www.h2o.ai/)

:::

::::


# Neural Networks {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

## What are neural networks? {data-background="../Backgrounds/reference.jpg"}

- The phrase *neural network* is thrown around almost like a buzz word
- *Neural networks* are actually a specific type class algorithms
    - There are many implementations with different primary uses

![](../Figures/analytics_ml.png){height="500px" fig-align="center"}

## What are neural networks? {data-background="../Backgrounds/reference.jpg"}

- Originally, the goal was to construct an algorithm that behaves like a human brain
    - Thus the name
- Current methods don't quite reflect human brains, however:
    1. We don't fully understand how our brains work, which makes replication rather difficult
    2. Most neural networks are constructed for specialized tasks (not general tasks)
    3. Some (but not all) neural networks use tools our brain may not have
        - I.e., **backpropogation** is <a href="https://www.frontiersin.org/articles/10.3389/fncom.2016.00094/full">potentially possible in brains</a>, but it is not pinned down how such a function occurs (if it does occur)

## What are neural networks? {data-background="../Backgrounds/reference.jpg"}

- Neural networks are a method by which a computer can learn from observational data
- In practice:
    - They were not computationally worthwhile until the mid 2000s
    - They have been known since the 1950s (perceptrons)
    - They can be used to construct algorithms that, at times, perform better than humans themselves
        - But these algorithms are often quite computationally intense, complex, and difficult to understand
    - Much work has been and is being done to make them more accessible

## Types of neural networks {data-background="../Backgrounds/default.jpg"}

- There are *a lot* of neural network types
    - See The <a href="http://www.asimovinstitute.org/neural-network-zoo/">"Neural Network Zoo"</a>
- Some of the more interesting ones which we will see or have seen:
    - RNN: Recurrent Neural Network
    - LSTM: Long/Short Term Memory
    - CNN: Convolutional Neural Network
    - DAN: Deep Averaging Network
    - GAN: Generative Adversarial Network
- Others worth noting
    - VAE (Variational Autoencoder): Generating *new* data from datasets
- Not in the Zoo, but of note:
    - <a href="http://jalammar.github.io/illustrated-transformer/">Transformer</a>: Networks with "attention"
        - From <a href="https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf">Attention is All You Need</a>

## CNN: Convolutional NN {data-background="../Backgrounds/neural-net.jpg"}

- Networks that excel at object detection (in images)
- Can be applied to other data as well
- Ex.: [Inception-v3](https://ai.googleblog.com/2016/03/train-your-own-image-classifier-with.html)

![](../Figures/NN_CNN.png){width="450px" fig-align="center"}

## DAN: Deep Averaging Network {data-background="../Backgrounds/neural-net.jpg"}

- DANs are simple networks that simply average their inputs
- Averaged inputs are then processed a few times
- These networks have found a home in NLP
    - Ex.: [Universal Sentence Encoder](https://tfhub.dev/google/universal-sentence-encoder/4)

![](../Figures/NN_DAN.png){width="675px" fig-align="center"}

## RNN: Recurrent NN {data-background="../Backgrounds/neural-net.jpg"}

- Recurrent neural networks embed a history of information in the network
    - The previous computation affects the next one
    - Leads to a *short term memory*
- Used for speech recognition, image captioning, anomaly detection, and many others
    - Also the foundation of LSTM
    - [SketchRNN](https://ai.googleblog.com/2017/04/teaching-machines-to-draw.html)<a href=""> ([live demo](https://magenta.tensorflow.org/assets/sketch_rnn_demo/index.html))

![](../Figures/NN_RNN.png){width="675px" fig-align="center"}

## LSTM: Long Short Term Memory {data-background="../Backgrounds/neural-net.jpg"}

- LSTM improves the *long term memory* of the network while explicitly modeling a *short term memory*
- Used wherever RNNs are used, and then some
    - Ex.: [Seq2seq](https://google.github.io/seq2seq/) (machine translation)

![](../Figures/NN_LSTM.png){width="600px" fig-align="center"}

## Transformer {data-background="../Backgrounds/neural-net.jpg"}

- Shares some similarities with RNN and LSTM: Focuses on attention
- Currently being applied to solve many types of problems
- Examples: BERT, GPT-3, XLNEt, [RoBERTa](https://app.inferkit.com/demo), ChatGPT

![](../Figures/Transformer.png){width="650px" fig-align="center"}

# Vector space models {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

## Motivating examples {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}

[![](../Figures/semantris.png){width="550px" fig-align="center"}](https://research.google.com/semantris/)

:::

::: {.column width="50%"}

[![](../Figures/talk_to_books.png){width="550px" fig-align="center"}](https://books.google.com/talktobooks/)

:::

::::



## What are "vector space models" {data-background="../Backgrounds/default.jpg"}

- Different ways of converting some abstract information into numeric information
    - Focus on maintaining some of the underlying structure of the abstract information
- Examples (in chronological order):
    - Word vectors:
        - [Word2vec](https://www.tensorflow.org/tutorials/representation/word2vec)
        - [GloVe](https://nlp.stanford.edu/projects/glove/)
    - Sentence vectors:
        - [Universal Sentence Encoder](https://tfhub.dev/google/universal-sentence-encoder-large/5)

## Word vectors {data-background="../Backgrounds/default.jpg"}

- Instead of coding individual words, encode word meaning
- The idea:
    - Our old way (encode words as IDs from 1 to N) doesn't understand relationships such as:
        - Spatial
        - Categorical
        - Grammatical (weakly when using stemming)
        - Social
        - etc.
    - Word vectors try to encapsulate all of the above
        - They do this by encoding words as a vector of different features

## Word vectors: Simple example {data-background="../Backgrounds/default.jpg"}

```{r, echo=F}
wv <- data.frame(words = c('dog', 'cat', 'Bill', 'turkey', 'Turkey', "Singapore"),
                 f_animal = c(0.5, 0.5, 0.1, 0.5, -0.5, -0.5),
                 f_people = c(0.3, 0.1, 0.9, -0.2, 0.1, 0.1),
                 f_location = c(-0.3, -0.3, -0.4, -0.3, 0.7, 0.8))
html_df(wv)
```

- The above is an idealized example
- Notice how we can tell apart different animals based on their relationship with people
- Notice how we can distinguish turkey (the animal) from Turkey (the country) as well

## What it retains: [word2vec](https://www.tensorflow.org/tutorials/representation/word2vec#visualizing_the_learned_embeddings) {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}

![](../Figures/word2vec_gender.png){width="300px" fig-align="center"}
![](../Figures/word2vec_tense.png){width="300px" fig-align="center"}

:::

::: {.column width="50%"}

![](../Figures/word2vec_country.png){width="400px" fig-align="center"}

:::

::::

> Relations are retained as vectors between points (distance + direction)

## What it retains: [GloVe](https://nlp.stanford.edu/projects/glove/) {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}

![](../Figures/glove_gender.jpg){width="350px" fig-align="center"}
![](../Figures/glove_grammar.jpg){width="350px" fig-align="center"}

:::

::: {.column width="50%"}

![](../Figures/glove_ceo.jpg){width="350px" fig-align="center"}
![](../Figures/glove_zip.jpg){width="350px" fig-align="center"}

:::

::::

## How to build word vectors {data-background="../Backgrounds/default.jpg"}

- Two ways:
    1. Word co-occurrence (like how LDA worked)
        - Global Vectors (GloVe) works this way
            - Available from the `{text2vec}` package
    2. Word order (using an NN)
        - word2vec works this way
            - Available from the `{rword2vec}` package
            - Uses a 2 layer neural network

## How does word order work? {data-background="../Backgrounds/default.jpg"}

> Infer a word's meaning from the words around it

![](../Figures/cbow.png){width="800px" fig-align="center"}

> Refered to as CBOW (continuous bag of words)

## How else can word order work? {data-background="../Backgrounds/default.jpg"}

> Infer a word's meaning by *generating* words around it

![](../Figures/skipgram.png){width="800px" fig-align="center"}

> Refered to as the Skip-gram model

## When are vector embeddings useful? {data-background="../Backgrounds/default.jpg"}

1. You care about the words used, by not stylistic choices
2. You want to crunch down a bunch of words into a smaller number of dimensions without running any bigger models (like LDA) on the text.

> Demo: [rmc.link/colab_w2v](https://rmc.link/colab_w2v)

# Understanding phrases (or larger) {data-background-video="../Backgrounds/section.mp4" data-background-video-loop="true"}

## Document vectors {data-background="../Backgrounds/default.jpg"}

- Document vectors work very similarly to word vectors
    - 1 added twist: a document/paragraph/sentence level factor variable
        - This is used to learn a vector representation of each text chunk
        - Generally learned simultaneously with the word vectors
- This is quite related to what we learned with LDA as well!
    - Both can tell us the topics discussed

## Universal Sentence Encoder (USE) {data-background="../Backgrounds/default.jpg"}

- We saw this briefly last week
    - This is the algorithm with less bias
- Focused on representing sentence-length chunks of text

![](../Figures/gmail_demo.gif){width="600px" fig-align="center"}

## A fun example of with USE {data-background="../Backgrounds/lab.jpg"}

- [Predict Shakespeare with Cloud TPUs and Keras](https://colab.research.google.com/github/tensorflow/tpu/blob/master/tools/colab/shakespeare_with_tpu_and_keras.ipynb)

![](../Figures/shakespeare_USE.png){width="1100px" fig-align="center"}

## Cavaet on using USE {data-background="../Backgrounds/lab.jpg"}

- One big caveat: USE only knows what it's trained on
    - Ex.: Feeding the same USE algorithm WSJ text

> Samsung Electronics Co., suffering a handset sales slide, revealed a foldable-screen smartphone that folds like a book and opens up to tablet size.  <span class="hi1">Ah, horror? I play Thee to her alone;<br>    And when we have withdrom him, good all.<br>      Come, go with no less through.<br><br>      Enter Don Pedres. A flourish and my money. I will tarry. Well, you do!<br><br>LADY CAPULET.<br>Farewell; and you are

## How does USE work? {data-background="../Backgrounds/reference.jpg"}

- USE is based on DAN and Transformer
    - Learns the meaning of sentences via words' meanings (which it also learns)
- Learn more: [Original paper](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46808.pdf) and [TensorFlow page](https://tfhub.dev/google/universal-sentence-encoder-large/5)
- In practice, it works quite well

![](../Figures/USE_ex.png){height="450px" fig-align="center"}

## Try it out! {data-background="../Backgrounds/lab.jpg"}

- Run on <a href="https://rmc.link/colab_use">Google Colab</a>
    - Python code
    - Just click the cells in order, and click run
    - Colab provides free servers to run the code on
        - It still takes a few minutes to run though

> Look for the comment `# Add your own messages here`. You can add custom messages to customize the figure and learn what USE understands

## Bringing this into accounting {data-background="../Backgrounds/case_digital.jpg" class="default case_digital"}

> Understanding how markets react to sentiment-language laden discussion in annual reports

- Crowley and Wong (2023), "Understanding Sentiment through Context"
- Data: Annual report MD&A sections
- Premise: Understand whether *positive* and *negative* discussion are reacted to differently by markets conditional on what is being discussed
- Why USE?: We use it to abstract away from word choice and cluster text by its meaning

> Result: Yes -- positive discussion is not always positive, and negative discussion is not always negative

## Other Transformer models {data-background="../Backgrounds/default.jpg"}

- Various **GPT** models (OpenAI)
    - Such as [ChatGPT](https://openai.com/blog/chatgpt) and [GPT-4](https://openai.com/research/gpt-4)
- [**LLAMA 2** (Facebook)](https://about.fb.com/news/2023/07/llama-2/) -- This one is open source
- [**PaLM 2** (Google)](https://ai.google/discover/palm2/)
    - The model underlying Google's BARD chat AI
- [**Claude v1** (Anthropic)](https://www.anthropic.com/index/introducing-claude)
- [**BERT** (Google)](https://blog.google/products/search/search-language-understanding-bert/)
    - Now used for Google Search in at least 70 languages




## What is a GPT model? {data-background="../Backgrounds/default.jpg"}

> A GPT model is a type of *Large Language Model* (**LLM**)

- Large: many parameters in the model (usually >1 billion)
- Language: the models are trained by seeing a large amount of written text
    - They infer everything from language
- Model: It's just an algorithm like everything else

> What does GPT mean? Generative Pre-trained Transformers

- Generative: It provides answers by generating an answer based on some latent space, as opposed to selecting answers it has previously seen
- Pre-trained: It's seen a lot of data already.  That does not preclude it from seeing more.
- Transformer: It's based on a *transformer* neural network

## What can ____-GPT do? {data-background="../Backgrounds/default.jpg"}

:::: {.columns}

::: {.column width="50%"}

**What can they do**

- Classify data based on a small number of examples
    - "Few shot learning"
- Provide answers in flexible/trainable formats
- Encode and decode language
- Pattern matching
- Images as language

:::

::: {.column width="50%"}

**What can they not do**

- Unless you train it yourself, it won't have much domain-specific knowledge
- Beat single-purpose SOTA algorithms on most tasks

:::

::::

## How do different GPT models vary? {data-background="../Backgrounds/default.jpg"}

> Context length -- the amount of text it can handle at once

- GPT-2: 2,048 tokens
    - ~2 single-spaced pages each for a question and response
- GPT-3: 4,096 tokens
- GPT-3.5: 4,096
- Chat-GPT: 4,096 tokens
- GPT-4: 8,096 or 32,384 tokens
    - The larger model can handle ~30 pages single spaced, each, for a question and response

## Let's build one! {data-background="../Backgrounds/default.jpg" .small}

> To build it, go to: [rmc.link/colab_gpt2](https://rmc.link/colab_gpt2)

:::: {.columns}

::: {.column width="50%"}

- This is a simple GPT
    - 12,656 parameters
    - 2 possible tokens
    - A context length of 3

:::

::: {.column width="50%"}

- As a comparison, GPT-2 has:
    - 1.5 billion parameters
    - 50,257 possible tokens
    - a context length of 2,048

:::

::::

::: {.callout-tip}
## How to interpret the network

The arrows show transition from a set of 3 characters (0 or 1) to the next. In this process, the left-most character is dropped, the remaining two characters shift left, and a new character is added to the right side.

:::

- Look for the following:
    1. That it encodes simple patterns in the data well
    2. That answers are effectively probabilistic
    3. Why hallucination occurs


# End Matter {data-background-video="../Backgrounds/section.mp4"}

## Discussion {data-background="../Backgrounds/group.jpg"}

> What creative uses for the techniques discussed today do you expect to see become reality in accounting in the next 3-5 years?

- Brainstorm with your group and try to come up with 1 good use for some technique discussed today
- Each group will be asked to share 1 use

## Recap {data-background="../Backgrounds/ending.jpg"}

Today, we:

- Learned formally what neural networks (NNs) are
- Discussed a variety of NN-based algorithms
- Saw uses for word and sentence vectors in a financial context

## Wrap up {data-background="../Backgrounds/ending.jpg"}

- For next week:
    - Work on the group project!
        - Definitely try to get a submission in on Kaggle
    - We'll keep talking about neural networks
        - A bit more theory
        - A lot more examples
        - Some real neural networks coded in **R**
- Survey on the class session at this QR code:

{{< qrcode https://rmc.link/420survey10 qrsurvey width=200 height=200 >}}

## Packages used for these slides {data-background="../Backgrounds/code.jpg" visibility="uncounted"}


- `{DT}`
- `{downlit}`
- `{kableExtra}`
- `{knitr}`
- `{plotly}`
- `{quarto}`
- `{revealjs}`
- `{tidyverse}`

## Generating Shakespeare {data-background="../Backgrounds/code.jpg" visibility="uncounted"}

```{python, eval=F}
seed_txt = 'Looks it not like the king?  Verily, we must go! '  # Original code
seed_txt = 'SCENE I. Elsinore. A platform before the Castle.\n\n Enter Francisco and Barnardo, two sentinels.\n\nBARNARDO.\nWho’s there?\n\nFRANCISCO.\nNay, answer me. Stand and unfold yourself.\n\nBARNARDO.\nLong live the King!\n\n'  # Hamlet
seed_txt = 'Samsung Electronics Co., suffering a handset sales slide, revealed a foldable-screen smartphone that folds like a book and opens up to tablet size.'  # WSJ article
# From: https://www.wsj.com/articles/samsung-unveils-foldable-screen-smartphone-1541632221
```

## NOTES {class="never"}


- The lesson is getting pretty heavy now?  Maybe drop some content?  But also elongate some content because it goes a bit fast if there aren't questions.

- Enron
- Videos & sound to show in class
    - Emergence of Locomotion Behaviours in Rich Environments: https://www.youtube.com/watch?v=hx_bgoTF7bs
    - Performance RNN: https://magenta.tensorflow.org/demos/performance_rnn/index.html#2|2,0,1,0,1,1,0,1,0,1,0,1|1,1,1,1,1,1,1,1,1,1,1,1|1,1,1,1,1,1,1,1,1,1,1,1|false
    - Multi-Agent Hide and Seek: https://www.youtube.com/watch?v=kopoLzvh5jY
