PPOL 670-01: Missing Data/Relational Data

class: center, middle, inverse, title-slide

.title[
# PPOL 670-01: Missing Data/Relational Data
]
.subtitle[
## Week 12
]
.author[
### Alexander Podkul, PhD
]
.date[
### Spring 2023
]

---

## Tonight's Outline 
.pull-left[
- The Next Few Weeks
- Final Project Presentations 
- Missing Data 
- Relational Databases 
- __Break__ 
- Coding Exercises
]
.pull-right[
<img src="sticker.png" width="80%" style="display: block; margin: auto;" />

]

---
## The Next Few Weeks

__Tonight__ (April 12):
- Problem Set #5 assigned

__Next Week__ (April 19): 
- Problem Set #5 due
- Assorted R packages(!)
- Advanced web scraping (time permitting)

__Last Week (😭)__ (April 26): 
- Presentation week

__Later Last Week__ (April 28): 
- Peer comments submitted

__Later__ (May 10): 
- Final project due

---
class: inverse, center, middle
## Final Project Presentations

---
## Final Project Presentations

For final project presentations, I want to be sure to cover two things this evening: 
1. Expectations about the presentation itself 
2. Tutorial on make slides in R Markdown (it's easy!)

---
## Final Project Presentations: Expectations

__The Minimum__:

Presenting a slide deck (in either .html or .pdf format) for about 7-8 minutes that introduces the project's research question (with any additional context), data source, modeling approach, and any preliminary findings.

--
__Optional additional areas to include__: 
- providing details on measurement
- any limitations that your work might face
- any clever data wrangling or cleaning that needed to be done

--
__Other tips and expectations__:
1. Does not need to show a "complete" report ("rough draft")
2. Highlighting the data, any models, and _interpretation_
3. No expectation of including any code
4. Visualizations `$\geq$` tables 
5. Please keep to 7-8 minutes

---
## Final Project Presentations: Tutorial on Slidemaking

(Tutorial in .rmd format downloadable on the course site.)

In R Studio
File --> New File --> R Markdown...

Select "Presentation"

Select either .html or .pdf option

---
class: inverse, center, middle
## Missing Data

---
## Missing Data

To start with an example: imagine we are using the familiar "life expectancy" data set. We can estimate a decision tree using our data such that:

---
## Missing Data
But imagine a scenario where we don't have all of our data collected. In particular, for some reason, we are only able to collect 30% of our data in Africa. When we re-estimate the model (using the same hyperparameters), it looks like:

---
## Missing Data

.pull-left[
__Missing data__ problems are one of the issues that are far more likely to happen in the real world than in classroom-ready data sets. Broadly, "missing data" can refer to unit nonresponse or item nonresponse; tonight we'll be focusing on the latter.

Whether or not we are actively considering missing data in our approaches, R (or whatever language we are using) is going to make decisions related to how to handle missing data. 
]

.pull-right[
<table class=" lightable-paper" style='font-family: "Arial Narrow", arial, helvetica, sans-serif; margin-left: auto; margin-right: auto;'>
 <thead>
  <tr>
   <th style="text-align:right;"> ID </th>
   <th style="text-align:left;"> X </th>
   <th style="text-align:right;"> Y </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:right;"> 1 </td>
   <td style="text-align:left;"> NA </td>
   <td style="text-align:right;"> 4 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 2 </td>
   <td style="text-align:left;"> Group B </td>
   <td style="text-align:right;"> 2 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 3 </td>
   <td style="text-align:left;"> Group A </td>
   <td style="text-align:right;"> 1 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 4 </td>
   <td style="text-align:left;"> Group A </td>
   <td style="text-align:right;"> 2 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 5 </td>
   <td style="text-align:left;"> Group B </td>
   <td style="text-align:right;"> 4 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 6 </td>
   <td style="text-align:left;"> Group B </td>
   <td style="text-align:right;"> 4 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 7 </td>
   <td style="text-align:left;"> NA </td>
   <td style="text-align:right;"> 3 </td>
  </tr>
  <tr>
   <td style="text-align:right;"> 8 </td>
   <td style="text-align:left;"> Group A </td>
   <td style="text-align:right;"> NA </td>
  </tr>
</tbody>
</table>
]

---
## Missing Data

For example, if we don't consider missing data and just move forward you'll notice there are some decisions that our models will make by default:

```r
mod <- lm(Y~X, data = d)
summary(mod)
```

```
## 
## Call:
## lm(formula = Y ~ X, data = d)
## 
## Residuals:
##       2       3       4       5       6 
## -1.3333 -0.5000  0.5000  0.6667  0.6667 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)
## (Intercept)   1.5000     0.7265   2.065    0.131
## XGroup B      1.8333     0.9379   1.955    0.146
## 
## Residual standard error: 1.027 on 3 degrees of freedom
##   (3 observations deleted due to missingness)
## Multiple R-squared:  0.5602,	Adjusted R-squared:  0.4136 
## F-statistic: 3.821 on 1 and 3 DF,  p-value: 0.1456
```

---
## Missing Data: Types

Before we discuss various solutions to working with missing data, we can first break down the issue off missing data to three types:

1. missing completely at random (MCAR)
2. missing at random (MAR)
3. missing not at random (MNAR)

---
## Missing Data: Types

Before we discuss various solutions to working with missing data, we can first break down the issue off missing data to three types:

1. __missing completely at random (MCAR)__
2. missing at random (MAR)
3. missing not at random (MNAR)

MCAR refers to the idea that the mechanism responsible for missing data is _completely random_, that is not related to other features of our data

Example: if we produced a random of countries to include in our analysis

---
## Missing Data: Types

Before we discuss various solutions to working with missing data, we can first break down the issue off missing data to three types:

1. missing completely at random (MCAR)
2. __missing at random (MAR)__
3. missing not at random (MNAR)

MAR refers to the idea that the mechanism responsible for missing data is related to other features within our data

Example: if literacy scores are missing but related to socioeconomic status

---
## Missing Data: Types

Before we discuss various solutions to working with missing data, we can first break down the issue off missing data to three types:

1. missing completely at random (MCAR)
2. missing at random (MAR)
3. __missing not at random (MNAR)__

MNAR refers to the idea that the mechanism responsible for missing data is related specifically to what we are aiming to measure

Example: if a person fails to show up to a workplace drug test because they recently did drugs

---
## Missing Data: Types

To borrow an example from Richard McElreath:

---
## Missing Data: Solutions

<table class=" lightable-paper" style='font-family: "Arial Narrow", arial, helvetica, sans-serif; margin-left: auto; margin-right: auto;'>
 <thead>
  <tr>
   <th style="text-align:left;"> Type </th>
   <th style="text-align:left;"> Issue </th>
   <th style="text-align:left;"> Possible Solutions </th>
  </tr>
 </thead>
<tbody>
  <tr grouplength="2"><td colspan="3" style="border-bottom: 1px solid;"><strong>Ignorable</strong></td></tr>
<tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> MCAR </td>
   <td style="text-align:left;"> Loss of Power </td>
   <td style="text-align:left;"> Listwise Deletion </td>
  </tr>
  <tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> MAR </td>
   <td style="text-align:left;"> Need to account for cause of missingness </td>
   <td style="text-align:left;"> Imputation </td>
  </tr>
  <tr grouplength="1"><td colspan="3" style="border-bottom: 1px solid;"><strong>Non-ignorable</strong></td></tr>
<tr>
   <td style="text-align:left;padding-left: 2em;" indentlevel="1"> MNAR </td>
   <td style="text-align:left;"> Produces biased parameter estimates </td>
   <td style="text-align:left;"> Improve dataset </td>
  </tr>
</tbody>
</table>

---
## Missing Data: Actual Solutions

How can we fix missing data?

1. __Go find the missing data__
2. Everything else...

There are many cases where we can't go back and fill in missing data points. For example, if we consider survey datasets, it's nearly impossible for us to recontact the respondent and get a full answer to the question she or he left blank and even if we did we would have to assume that her or his answer would be the same as it would have been when the survey first fielded.

But in other settings, we might be able to fill in missing data, especially after better understanding the core reasons for missingness. For example, in the state dataset for the data project, there may have been reasons a researcher was unable to find historical data which may be available. By understanding why the data is missing you can better assess whether that data can be added to the data file. (though, as always, it's important to note how and when you're amending established datasets, if it's not an original dataset.)