---
title: "Looking at data"
date: "8/25/2017"
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

Let's take a look at some actual data and talk carefully about how we might view and describe it. This information correlates closely with chapter 2 and a little of chapter 3 of our textbook.

## Our class data

Let's begin with our classroom data. I've got that data recorded in [a CSV file right here](https://www.marksmath.org/data/class_data_Fall2017.csv). If you'd like to know how I scraped the data off of the webpage, you can [read about the process](ReadingClassData.html).

Written out in a table, the first few rows looks like so:

```{r, echo=F}
library(knitr)
class_df = read.csv('https://www.marksmath.org/data/class_data_Fall2017.csv')
kable(head(class_df))
```


We have a data frame with three columns - corresponding to three *variables*:

 * Gender: A nominal, categorical variable
 * Age: A discrete, numerical variable, and
 * Height: A continuous, numerical variable.

The major different types of data, categorical and numerical, require different types of visualization.

### A look at our one categorical variable

We've got just one categorical variable: Gender. *Bar plots* and *pie charts* are fundamental tools for visualizing this type of simple categorical data.

#### A pie chart

According to the 56 data entries currently posted, there are 17 men and 39 women enrolled in the class. (That includes my daughter's fake entry.) A common way to visualize relative amounts of categorical data *when there are only two categories* is with a pie chart. Here is the pie chart comparing the number of men and women in our class.

```{r, echo=F}
men = subset(class_df, Gender == 'm')
women = subset(class_df, Gender == 'f')
n_men = length(men$Age)
n_women = length(women$Age)
# c(n_men,n_women)
pie(c(n_men, n_women), labels = c("men","women"))
```

I emphasize that *relative proportion of two categories* like this is about the only situation where I would use a pie chart. Pie charts are generally poor when it comes to working with more categories. We'll take a closer look at this after we get data with more categories.

#### A bar plot

Another way (often better) way to visualize categories is with a bar plot:

```{r, echo=F}
barplot(c(n_men, n_women),names.arg = c("m","w"))
```

### A look at our numerical variables

We have two numerical variables: Age and Height. *Histograms* and *box plots* (or box and whisker plots) are two standard ways to visualize a list of numeric data.

#### A histogram

Let's take a look at a *histogram* of the heights (after removing my 8 year old daughter). The idea is to count how many observations lie in a collection of equally spaced intervals. A picture gets the idea across.

```{r, echo=F}
hist(subset(class_df, Age>10)$Height, col = 'gray', main='',
  xlab = 'Height'
)
```

The basic shape that we see here (heavy in the middle and lighter on the ends is common). Sometimes this is called a *bell shape* and indicates a *normal distribution*. While normally distributed data is common, not all data is normally distributed. I suppose there's no reason to think that age in a collge class should be normally distributed.


```{r, echo=F}
hist(subset(class_df, Age>10)$Age, col = 'gray', main='',
  xlab = 'Age'
)
```

#### A box plot

We briefly met box plots in our first day demo. Here's a look at a box plot for our class heights.

```{r, echo=F}
par(pin=c(4,1))
boxplot(class_df$Height, horizontal = T)
```

Any dots that we see are *outliers*. The thick line in the middle is the *median* and the two lines on either side of that represent the *quartiles*. The dashes at the end of the lines extending out of the box are the max and min, excluding the outliers. We'll talk a little more carefully about all of those in a bit.


## Another dataset - The Titanic

In chapter 2, the text book talks a lot about the Titanic. There is a famous data set for this disaster which I have [stored on my webspace](https://www.marksmath.org/data/titanic.csv). The first couple of rows look something like so:

```{r, echo=F}
library(knitr)
titanic_df = read.csv('https://www.marksmath.org/data/titanic.csv')
kable((head(titanic_df, 2)))
```

There are 2200 rows like that. The fields are:

  * Class:
    * 0  (for Crew)
    * 1  (for First)
    * 2  (for Second)
    * 3  (for Third)
  * Gender:
    * m
    * f
  * Age:
    * a  (for Adult)
    * c  (for Child)
  * Survived:
    * y 
    * n

Thus, what we have here are four categorical variables. Class is an example of an ordinal variable; the others are nominal, though it still helps to see them in context.

Looking at the complete data table is pretty much useless. We often look at various summary tables. I suppose the most logical is to count how many lived and how many did not.

```{r, echo=F}
table(titanic_df$Survived)
```
We can convert this to a relative frequency table by dividing through by the total number of people.

```{r, echo=F}
n = length(titanic_df$Age)
table(titanic_df$Survived)/n
```

More interesting is a contingency table that compares two of the variables.

```{r, echo=F}
addmargins(table(titanic_df$Class, titanic_df$Survived))
```

There are a number of ways to translate a contingency table into a relative frequency table.

  * Divide the rows by their sums
  * Divide the columns by their sums
  * Divide everything by the total sum

Here's a relative frequency table of the first type.

```{r, echo=F}
tbl = table(titanic_df$Class, titanic_df$Survived)
for(i in 1:4) {
  tbl[i,] = tbl[i,]/sum(tbl[i,])
}
addmargins(tbl,2)
```

We see quite clearly the dependence of survial on class.

I'm a big fan of Mosaic plots, which are geometric realizations of the third type of relative frequency table.

```{r, echo=F}
mosaicplot(table(titanic_df$Class, titanic_df$Survived))
```

I'm not a big fan of the pie chart. Here's a pie chart for class:

```{r, echo=F}
pie(table(titanic_df$Class))
```

And *please* don't **ever** make a 3D pie chart.

```{r, echo=F}
library(plotrix)
pie3D(table(titanic_df$Class))
```

Here's the only way that a pie chart is OK:

<img src="real_pie_chart.jpg" /><br />
[source](https://www.quora.com/When-are-pie-charts-better-for-data-than-bar-graphs-and-vice-versa)

---









