---
title: Introduction
author: David Kane
tutorial:
  id: introduction
output:
  learnr::tutorial:
    progressive: yes
    allow_skip: yes
runtime: shiny_prerendered
description: Tutorial for Introduction
---

```{r setup, include = FALSE}
library(learnr)
library(tutorial.helpers)
library(tidyverse)

knitr::opts_chunk$set(echo = FALSE)
knitr::opts_chunk$set(out.width = '90%')
options(tutorial.exercise.timelimit = 60, 
        tutorial.storage = "local") 

```

```{r copy-code-chunk, child = system.file("child_documents/copy_button.Rmd", package = "tutorial.helpers")}
```

```{r info-section, child = system.file("child_documents/info_section.Rmd", package = "tutorial.helpers")}
```

<!-- Took students 42 minutes on average, so could add a lot more material. 60 minutes is a good target. -->

<!-- Add glimpse(), summary(), slice_sample(), print() and other commands for looking at a data set. Also drop_na(). -->

<!-- Perhaps add a little introduction to R scripts. That way, we could use them in the first class. -->

<!-- Fix introduction and summary. -->

<!-- Should the data pipe come before the plot? -->

<!-- Add drop_na() since we often use that in the first week of the course. -->

<!-- Make this tutorial better. Make sure there are no mistakes. Make sure it is clear. Drop a bunch of knowledge from Chapter 1. But also do some reminding of stuff from Getting Started from tutorial.helpers. -->

<!-- Why don't links open if you pull out the tutorial into its own window? Mention this somewhere here. In Introduction, Make more clear that you can pull out the tutorial into its own window. But is that really a good idea? Explain to students how to move border pane around. Do this in tutorial.helpers?  In the introduction to the setting of RStudio, there could be more information on how to manipulate the size of each pane (ex. to view the content of the tutorial, to have more space for coding). -->

<!-- Need to make sure the early tutorials cover things like scale_x_log10(labels = scales::comma_format()) since we use it in class in weeks 1 and 2. -->

<!-- Maybe rethink the background questions in Prerequisites. Could have more? (Like who to have in your breakout room. How much CS or Stats background they have. What city are they in.) -->


## Introduction
### 

This tutorial covers the [Introduction](https://r4ds.hadley.nz/intro.html) from [*R for Data Science (2e)*](https://r4ds.hadley.nz/) by Hadley Wickham, Mine Çetinkaya-Rundel, and Garrett Grolemund. 

The promise we make to teachers is that these tutorials will ensure that your students type in 90% of the code which appears in the associated chapter. This chapter has very little R code. So, in the spirit of getting started, we include some of our own.

### 

This tutorial assumes that you have already completed the Getting Started tutorial in the [**tutorial.helpers**](https://ppbds.github.io/tutorial.helpers/) package. If you haven't, do so now. It is quick!


## What you will learn
### 

In this tutorial, we will only work within the Exercise Code Blocks. However, you can also put the commands into the Console tab in the lower left. 


### Exercise 1

Data science is a vast field, and there’s no way you can master it all by reading a single book. This book aims to give you a solid foundation in the most important tools and enough knowledge to find the resources to learn more when necessary. Our model of the steps of a typical data science project looks something like this:

```{r what-you-will-learn-1}
knitr::include_graphics("images/base.png")
```

### 

### Exercise 2

Before you start doing data science, you must load the packages you are going to use. Use the function `library()` to load the **tidyverse** package.

```{r what-you-will-learn-2, exercise = TRUE}

```

```{r what-you-will-learn-2-hint-1, eval = FALSE}
library(tidyverse)
```

```{r what-you-will-learn-2-test, include = FALSE}
library(tidyverse)
```

### 

Nothing is returned, which is often the case with R code. But note the check mark which has appeared next to "Exercise 2" above. This indicates that you have submitted your answer.

### 

Note that "library" and "package" mean the same things in R. We have different words for historical reasons. However, only the `library()` command will load a package/library, giving us access to the functions and data which it contains.

### 

Looking at the graphic above, the first step in a data science project is to import your data into R. This typically means that you take data stored in a file, database, or web application programming interface (API) and load it into a data frame in R. If you can’t get your data into R, you can’t do data science on it!

### Exercise 3

If you are ever stuck while coding, R has *help pages*. Let's say we want to know what the function `sqrt()` does. Open the help page for `sqrt()` by typing `?sqrt` below.

```{r what-you-will-learn-3, exercise = TRUE}

```

```{r what-you-will-learn-3-hint-1, eval = FALSE}
?sqrt
```

### 

After importing your data, it is a good idea to **tidy** it. Tidying your data means storing it in a consistent form that matches the semantics of the dataset with how it is stored. In brief, when your data is tidy, each column is a variable and each row is an observation. Tidy data is important because the consistent structure lets you focus your efforts on answering questions about the data, not fighting to get the data into the right form for different functions.


### Exercise 4

Type `?diamonds` to look up the help page for the `diamonds` tibble from the **ggplot2** package, which is one of the core packages in the *Tidyverse*.

```{r what-you-will-learn-4, exercise = TRUE}

```

```{r what-you-will-learn-4-hint-1, eval = FALSE}
?diamonds
```

Copy/paste the Description from the help page into the box below.

```{r looking-at-data-4-b}
question_text(NULL,
	answer(NULL, correct = TRUE),
	allow_retry = TRUE,
	try_again_button = "Edit Answer",
	incorrect = NULL,
	rows = 3)
```

### 

You can find help about an entire package with `help(package = "ggplot2")`. It is confusing, but unavoidable, that package names are sometimes unquoted, as in `library(ggplot2)`, and sometimes quoted, as in `help(package = "ggplot2")`. If one does not work, try the other.


### Exercise 5

Assign the value of `sqrt(144)` to the variable `x`. Remember to use the assignment operator `<-`.

```{r what-you-will-learn-5, exercise = TRUE}

```

```{r what-you-will-learn-5-hint-1, eval = FALSE}
x <- sqrt(144)
```

```{r what-you-will-learn-5-test, include = FALSE}
x <- sqrt(144)
```

### 

Once you have tidy data, a common next step is to **transform** it. Transformation includes narrowing in on observations of interest (like all people in one city or all data from the last year), creating new variables that are functions of existing variables (like computing speed from distance and time), and calculating a set of summary statistics (like counts or means). Together, tidying and transforming are called **wrangling** because getting your data in a form that’s natural to work with often feels like a fight!

### Exercise 6

Data frames, also referred to as "tibbles", are spreadsheet-type data sets. 

### 

In data science, it's always a good idea to explore the data before you use it. You can simply examine a data set by typing its name. Let's examine the contents of `diamonds` data set. Type `diamonds` in the line below.

```{r what-you-will-learn-6, exercise = TRUE}

```

```{r what-you-will-learn-6-hint-1, eval = FALSE}
diamonds
```

```{r what-you-will-learn-6-test, include = FALSE}
diamonds
```

### 

**Visualization** is a fundamentally human activity. A good visualization will show you things you did not expect or raise new questions about the data. A good visualization might also hint that you’re asking the wrong question or that you need to collect different data. Visualizations can surprise you, but they don’t scale particularly well because they require a human to interpret them.


### Exercise 7

When we type the name of a tibble in the R Console, we are implicitly using the `print()` function. Try it by using `print()` on `diamonds`.

```{r what-you-will-learn-7, exercise = TRUE}

```

```{r what-you-will-learn-7-hint-1, eval = FALSE}
print(...)
```

```{r what-you-will-learn-7-test, include = FALSE}
print(diamonds)
```

This produces the same result as just typing `diamonds`. But `print()` is more flexible, allowing us to specify different numbers of rows and columns. You can use the `width` argument to change how many columns appear. `print(diamonds, width = Inf)` will cause all columns to be printed.

### Exercise 8

Print out the first three rows of `diamonds`. Note that the number of rows printed is controlled by the `n` argument.

```{r what-you-will-learn-8, exercise = TRUE}

```

```{r what-you-will-learn-8-hint-1, eval = FALSE}
print(diamonds, n = 3)
```

```{r what-you-will-learn-8-test, include = FALSE}
print(diamonds, n = 3)
```

### 

**Models** are complementary tools to visualization. Once you have made your questions sufficiently precise, you can use a model to answer them. Models are fundamentally mathematical or computational tools, so they generally scale well. Even when they don’t, it’s usually cheaper to buy more computers than it is to buy more brains! But every model makes assumptions, and by its very nature, a model cannot question its own assumptions. That means a model cannot fundamentally surprise you.


### Exercise 9

Now use `glimpse()` to examine the `diamonds` data set. To learn more about the `glimpse()` function, run `?glimpse`.

```{r what-you-will-learn-9, exercise = TRUE}

```

```{r what-you-will-learn-9-hint-1, eval = FALSE}
glimpse(diamonds)
```

```{r what-you-will-learn-9-test, include = FALSE}
glimpse(diamonds)
```

### 

`glimpse()` displays columns running down the page and the data running across across. Note how the "type" of each variable is listed next to the variable name. For example, `price` is listed as `<int>`, meaning that it is an integer variable.

`view()` is another useful function, but, because it is interactive, we should not use it within a tutorial. 

### Exercise 10

Run `summary()` on `diamonds`.

```{r what-you-will-learn-10, exercise = TRUE}

```

```{r what-you-will-learn-10-hint-1, eval = FALSE}
summary(...)
```

```{r what-you-will-learn-10-test, include = FALSE}
summary(diamonds)
```

### 

Referring again to our graphic, the last step of data science is **communication**, an absolutely critical part of any data analysis project. It doesn’t matter how well your models and visualization have led you to understand the data unless you can also communicate your results to others.

### Exercise 11

*Code comments* are text placed after a `#` symbol. Nothing will be run after a `#` symbol, which is useful if you want to write human readable comments in your code.

### 

Press "Run Code." Afterwards, add the `#` and re-run the chunk. You should no longer see a result.

```{r what-you-will-learn-11, exercise = TRUE}
sqrt(144)
```

Surrounding all these tools is **programming**. Programming is a cross-cutting tool that you use in nearly every part of a data science project. You don’t need to be an expert programmer to be a successful data scientist, but learning more about programming pays off because becoming a better programmer allows you to automate common tasks and solve new problems with greater ease.



## How this book is organized
### 

The previous description of the tools of data science is organized roughly according to the order in which you use them in an analysis (although, of course, you’ll iterate through them multiple times). In our experience, however, learning data importing and tidying first is suboptimal because, 80% of the time, it’s routine and boring, and the other 20% of the time, it’s weird and frustrating. That’s a bad place to start learning a new subject!

Instead, we’ll start with visualization and transformation of data that’s already been imported and tidied. 

### 

Let's create the following scatterplot from the `mpg` dataset, which provides measurements of attributes from various car models.

```{r scat-p}
scat_p <- ggplot(data = mpg, 
                 mapping = aes(x = displ, 
                               y = hwy)) +
  geom_point(alpha = 0.5, color = "steelblue") +
  labs(title = "Measurements for Engine Displacement and Highway Fuel Efficiency of Select Car Models",
       subtitle = "Cars with greater engineer displacement are less fuel efficient",
        x = "Engine Displacement (L)",
        y = "Highway Efficiency (mpg)", 
       caption = "EPA (2008)")

scat_p
```

### Exercise 1

First, let's `glimpse()` the `mpg` data set. Looking at the axis titles above, can you determine what the names are for the two variables we will use?

```{r how-this-book-is-organized-1, exercise = TRUE}

```

```{r how-this-book-is-organized-1-hint-1, eval = FALSE}
glimpse(...)
```

```{r how-this-book-is-organized-1-test, include = FALSE}
glimpse(mpg)
```

### 

`glimpse()` is most effective when you want to see both all the variables in a data set and many observations.

We are going to use `displ` and `hwy` to create the plot.

### Exercise 2

Run `ggplot()`, setting `data` equal to `mpg`.

```{r how-this-book-is-organized-2, exercise = TRUE}

```

```{r how-this-book-is-organized-2-hint-1, eval = FALSE}
ggplot(data = ...)
```

```{r how-this-book-is-organized-2-test, include = FALSE}
ggplot(data = mpg)
```

`ggplot()` initializes a ggplot object. Your output should be an empty screen.


### Exercise 3

The first argument to `ggplot()` is `data`, as above. The second argument is `mapping`. Set the `mapping` to equal `aes()`, which is the "aesthetics" function for plotting.

```{r how-this-book-is-organized-3, exercise = TRUE}

```

<button onclick = "transfer_code(this)">Copy previous code</button>

```{r how-this-book-is-organized-3-hint-1, eval = FALSE}
ggplot(data = mpg, mapping = ...())
```

```{r how-this-book-is-organized-3-test, include = FALSE}
ggplot(data = mpg, mapping = aes())
```

This produces the same blank canvas as above. We need to specify some arguments to `aes()` in order to generate the plot.

### Exercise 4

The two most important arguments in `aes()` are `x` and `y`. Set `x` equal to `displ`. Set `y` equal to `hwy`.


```{r how-this-book-is-organized-4, exercise = TRUE}

```

<button onclick="transfer_code(this)">Copy previous code</button>

```{r how-this-book-is-organized-4-hint-1, eval = FALSE}
ggplot(data = mpg, 
       mapping = aes(x = ...,
                     y = ...))
```

```{r how-this-book-is-organized-4-test, include = FALSE}
ggplot(data = mpg, 
       mapping = aes(x = displ, 
                     y = hwy))
```

### 

Anything included in `aes()` brings some information from the data in our tibble onto the graph. In this case, R knows that `displ` (a measure of the size of a car's engine) goes on the x axis and `hwy` (miles per gallon for highway driving) goes on the y-axis. 

R can also see the range of values in `mpg` for both `displ` and `hwy`, thereby determining the range of values which the axes should cover.

### Exercise 5

Let's now add the layer `geom_point()`. Steps within a series of plotting commands are connected by plus signs (`+`).

```{r how-this-book-is-organized-5, exercise = TRUE}

```

<button onclick="transfer_code(this)">Copy previous code</button>

```{r how-this-book-is-organized-5-hint-1, eval = FALSE}
Remember when you add a layer you use `+`. 
```

```{r how-this-book-is-organized-5-hint-2, eval = FALSE}
ggplot(data = mpg, 
       mapping = aes(x = displ,
                     y = hwy)) +
  geom_...()
```

```{r how-this-book-is-organized-5-test, include = FALSE}
ggplot(data = mpg, 
       mapping = aes(x = displ,
                     y = hwy)) +
  geom_point()
```

### 

`geom_*` functions (such as `geom_point()`) add additional layers to the base ggplot. This allows us to create a graphic piece-by-piece.

### Exercise 6

The code above uses the `mpg` tibble to create a scatterplot that displays 126 points, however, it visualizes a data set that contains 234 points. Because many points share the same values, this causes individual data points to be hidden behind other points. This is also known as **overplotting.**

One method to fight overplotting is to make each point **semi-transparent**. Change the transparency of the points by setting `alpha` equal to `0.5` within the call to `geom_point()`.

```{r how-this-book-is-organized-6, exercise = TRUE}

```

<button onclick="transfer_code(this)">Copy previous code</button>

```{r how-this-book-is-organized-6-hint-1, eval = FALSE}
ggplot(data = mpg, 
       mapping = aes(x = displ, 
                     y = hwy)) +
  geom_point(alpha = ...)
```

```{r how-this-book-is-organized-6-test, include = FALSE}
ggplot(data = mpg, 
       mapping = aes(
         x = displ,
         y = hwy)) +
  geom_point(alpha = .5)
```

### 

`alpha` only changes the appearance of the graph and does not add new information from the data. Thus, this argument is within the `geom` and is not nested within a call to `aes()`.

### Exercise 7

Now, also within `geom_point()`, set `color` equal to `"steelblue"`.

```{r how-this-book-is-organized-7, exercise = TRUE}

```

<button onclick="transfer_code(this)">Copy previous code</button>

```{r how-this-book-is-organized-7-hint-1, eval = FALSE}
ggplot(data = mpg, 
       mapping = aes(x = displ, 
                     y = hwy)) +
  geom_point(alpha = ..., 
             color = "...")
```

```{r how-this-book-is-organized-7-test, include = FALSE}
ggplot(data = mpg, 
       mapping = aes(
         x = displ,
         y = hwy)) +
  geom_point(alpha = .5, 
             color = "steelblue")
```

### 

R has 657 (built in color names)[https://www.nceas.ucsb.edu/sites/default/files/2020-04/colorPaletteCheatsheet.pdf?page=3]. Like `alpha`, `color` is not changing the data so the argument is within `geom`.

### Exercise 8

<!-- Could split this into 5 exercises. . . -->

Now, use `labs()` to add labeling to the graph. Set the `title` to `"Measurements for Engine Displacement and Highway Fuel Efficiency of Select Car Models"` and the `subtitle` to `"Cars with greater enginer displacement are less fuel efficient"`. To label the axes, set `x` to `"Engine Displacement (L)"` and `y` to `"Highway Efficiency (mpg)"`. Set the `caption` to `"EPA (2008)"`.

The caption is generally used to credit the relevant data sources.

```{r how-this-book-is-organized-8, exercise = TRUE }

```

<button onclick="transfer_code(this)">Copy previous code</button>

```{r how-this-book-is-organized-8-hint-1, eval = FALSE}
Remember to add a layer with the function labs()
```

```{r how-this-book-is-organized-8-hint-2, eval = FALSE}
... +
  labs(title = "...",
       subtitle = "...",
       x = "...",
       y = "...",
       caption = "...") 
```

Reminder: This is what our graph should look like.

```{r show-scat-p}
scat_p
```

## What you won’t learn
### 

[*R for Data Science (2e)*](https://r4ds.hadley.nz/) and these associated tutorials cover a lot of material. But we can't cover everything. In particular, the book and these tutorials do not cover modeling, big data, or other programming languages like Python and Julia.


## Prerequisites
### 

If you are running this tutorial, then you probably already know about R and RStudio. If, for some reason you don't, this [Getting Started chapter](https://ppbds.github.io/primer/getting-started.html) is the best place to start.

If you plan to do several of the tutorials in this package, you may find it useful to install all the necessary packages. Simple copy/paste this code to Console.


```
install.packages(
  c("arrow", "babynames", "curl", "duckdb", "gapminder", 
    "ggrepel", "ggridges", "ggthemes", "hexbin", "janitor", "Lahman", 
    "leaflet", "maps", "nycflights13", "openxlsx", "palmerpenguins", 
    "repurrrsive", "tidymodels", "writexl")
  )
```

<!-- Is the above really all the packages? What about the database ones? Or jsonlite? -->

Below are two background questions which are sometimes used by instructors in organizing their breakout rooms.

### Exercise 1

How old are you?

```{r prerequisites-1}
question_text(NULL,
	answer(NULL, correct = TRUE),
	allow_retry = TRUE,
	try_again_button = "Edit Answer",
	incorrect = NULL,
	rows = 1)
```

### Exercise 2

What is your highest level of education? (Just completed 10th grade. Sophomore in college. Et cetera.)

```{r prerequisites-2}
question_text(NULL,
	answer(NULL, correct = TRUE),
	allow_retry = TRUE,
	try_again_button = "Edit Answer",
	incorrect = NULL,
	rows = 1)
```

## Running R code
### 

Although the *Tidyverse* includes hundreds of commands, the most important are `filter()`, `select()`, `arrange()`, `mutate()`, and `summarize()`. Whenever you face a new problem, try to think about which one of these commands might be a good way to start.


### Exercise 1

Let's warm up by examining the `gss_cat` tibble from the **forcats** package. Since **forcats** is a core **tidyverse** package, you have already loaded it. Type `gss_cat` and hit "Run Code."

```{r running-r-code-1, exercise = TRUE}

```

```{r running-r-code-1-hint-1, eval = FALSE}
gss_cat
```

```{r running-r-code-1-test, include = FALSE}
gss_cat
```

### 

Whenever we print a tibble, the number of rows and columns is displayed at the top:

````
A tibble: 21,483 × 9
````

You can also see the variable type under each of the column names.

### Exercise 2

Run `?forcats::gss_cat` at the Console. This should work even if you have not loaded the **forcats** package. The double colon --- `::` --- notation allows us to access the inside of a package even if we have not loaded it.

Copy/paste the Description.

```{r running-r-code-2}
question_text(NULL,
	answer(NULL, correct = TRUE),
	allow_retry = TRUE,
	try_again_button = "Edit Answer",
	incorrect = NULL,
	rows = 3)
```

### 

Throughout these tutorials, we use a consistent set of conventions to refer to code:

Functions are displayed in a code font and followed by parentheses, like `sum()` or `mean()`.

Other R objects (such as data or function arguments) are in a code font, without parentheses, like `flights` or `x`.

Sometimes, to make it clear which package an object comes from, we’ll use the package name followed by two colons, like `dplyr::mutate()` or `nycflights13::flights`. This is also valid R code.

### Exercise 3

Recall that the `+` sign is used to "chain" different pieces of plot creation code together. When doing data analysis, we use the "pipe" symbol --- `|>` --- to do the same thing between different pieces code which manipulate the data.

As a simple example, "pipe" the `gss_cat` tibble to the `print()` command.  


```{r running-r-code-3, exercise = TRUE}

```

```{r running-r-code-3-hint-1, eval = FALSE}
gss_cat |> 
  print()
```

```{r running-r-code-3-test, include = FALSE}
gss_cat |> 
  print()
```

### 

Note the language. We write "pipe this to that." That is, we pipe the `gss_cat` tibble to the `print()` command. This accomplishes the same effect as simply running `print(gss_cat)`, but allows us to string together several commands in a row.


### Exercise 4

Pipe `gss_cat` to `filter()` using the `|>`.  Within `filter()`, use the argument `age > 88`. In the same way that the `+` sign is used to "chain" different pieces of plot creation code together, the "pipe" --- `|>` --- is used to chain different pieces of data analysis code together.

```{r running-r-code-4, exercise = TRUE}

```

```{r running-r-code-4-hint-1, eval = FALSE}
gss_cat |> 
  filter(... > 88)
```

```{r running-r-code-4-test, include = FALSE}
gss_cat |> 
  filter(age > 88)
```

### 

This workflow --- in which we pipe a tibble to a function, which then outputs another tibble, which we can then pipe to another function, and so on --- is very common in R programming.

The resulting tibble has the same number of columns --- `filter()` only affects the rows --- as `gss_cat` but many fewer rows, because there are only 150 people in the data older than 88.

### Exercise 5

Continue the pipe with `select()`, using the argument `age, marital, race, relig, tvhours`.


```{r running-r-code-5, exercise = TRUE}

```

<button onclick = "transfer_code(this)">Copy previous code</button>

```{r running-r-code-5-hint-1, eval = FALSE}
... |> 
  select(age, ..., race, ..., tvhours)
```

```{r running-r-code-5-test, include = FALSE}
gss_cat |> 
  filter(age > 88) |> 
  select(age, marital, race, relig, tvhours)
```

### 

Note how the Hint only gives the most recent line of the pipe. Because `select()` does not affect the rows, we have the same number as after `filter()`. But we only have 5 columns now, consistent with what we told `select()` to do.

### Exercise 6

Use the "Copy previous code" button to place a copy of the pipe from the last question into the Exercise Code Block. Extend the pipe with `arrange()`, using `desc(tvhours)` as the argument.

```{r running-r-code-6, exercise = TRUE}

```

<button onclick = "transfer_code(this)">Copy previous code</button>

```{r running-r-code-6-hint-1, eval = FALSE}
... |> 
  arrange(desc(...))
```

```{r running-r-code-6-test, include = FALSE}
gss_cat |> 
  filter(age > 88) |> 
  select(age, marital, race, relig, tvhours) |> 
  arrange(desc(tvhours))
```

### 

Got to respect someone who watches TV 18 hours a day!



## Summary
### 

This tutorial covered the [Introduction](https://r4ds.hadley.nz/intro.html) from [*R for Data Science (2e)*](https://r4ds.hadley.nz/) by Hadley Wickham, Mine Çetinkaya-Rundel, and Garrett Grolemund. 

```{r download-answers, child = system.file("child_documents/download_answers.Rmd", package = "tutorial.helpers")}
```
