1 The Grammar of Graphics

1.1.1 Explanatory Visualization

Explanatory visualization is polished, publication-quality, and interpretable:

• Meant to be consumed by broad, non-specialist audiences
• Takes significant time and iterations to perfect
• Conveys one or two “big ideas”, each

Just note the length of the code and its result. What idea does this plot convey?

library(ggplot2)

ggplot(read_csv(url), aes(x = val, y = myr)) +
  geom_jitter(alpha = 0.3, color = "tomato") +
  geom_smooth(method = "lm", se = FALSE, 
              color = "grey50", lwd = 1, alpha = 0.3) +
  facet_wrap(~ elm) +
  labs(title = "Annual income vs. Holland personality scores", 
       x = "Personality Score",
       y = "Income (K)",
       caption = "Sources: US DOL O*NET & BLS") +
  scale_y_continuous(labels = c("0", "50", "100", "150", "$200")) +
  theme_minimal()

To experiment with Holland codes and predicted income, check it out in Shiny.

1.1.2 Exploratory Visualization

Exploratory visualization is “quick and dirty” and intends to discover.

• Meant to be consumed by yourself, colleagues, or other specialists
• Created quickly, with no polish, refinement, or audience in mind
• Can convey many ideas, or none - that’s why you do it!

This code is manageable! Observe a simple call to ggpairs() and the iris dataset.

library(ggplot2)
library(GGally)

ggpairs(data = iris, 
        aes(color = Species))

Hopefully, you wouldn’t publish this. But we can quickly find patterns in a pairs plot:

• We can see positive correlations between sepal and petals for two species
• We observe that “Setosa” have thinner, longer sepals compared to others
• We also observe that “Versicolor” has the least variation in size

Such visual exploration can help refine hypotheses before analysis begins!

1.1.3 Do I have to Visualize?

Exploratory viz is a key component in exploratory data analysis, or EDA.

Failing to visually explore your data can get you in hot water. Let’s try it!

Observe the following data frame containing four data sets.

Variable x1 corresponds to y1, x2 to y2, and so forth:

library(datasets)

anscombe

##    x1 x2 x3 x4    y1   y2    y3    y4
## 1  10 10 10  8  8.04 9.14  7.46  6.58
## 2   8  8  8  8  6.95 8.14  6.77  5.76
## 3  13 13 13  8  7.58 8.74 12.74  7.71
## 4   9  9  9  8  8.81 8.77  7.11  8.84
## 5  11 11 11  8  8.33 9.26  7.81  8.47
## 6  14 14 14  8  9.96 8.10  8.84  7.04
## 7   6  6  6  8  7.24 6.13  6.08  5.25
## 8   4  4  4 19  4.26 3.10  5.39 12.50
## 9  12 12 12  8 10.84 9.13  8.15  5.56
## 10  7  7  7  8  4.82 7.26  6.42  7.91
## 11  5  5  5  8  5.68 4.74  5.73  6.89

At a glance, they look like the have some similaries!

Check It: Let’s perform a few statistical EDA functions on each subset, 1-4.

What do you notice when we figure out and organize:

• Average of all X and Y values with mean()
• Variance of all X and Y values with var()
• Correlation between X and Y for all sets with cor()
• Linear regression coefficients between X and Y with lm()

##       x_mean   y_mean x_var    y_var    correl     coeff
## Set 1      9 7.500909    11 4.127269 0.8164205 0.5000909
## Set 2      9 7.500909    11 4.127629 0.8162365 0.5000000
## Set 3      9 7.500000    11 4.122620 0.8162867 0.4997273
## Set 4      9 7.500909    11 4.123249 0.8165214 0.4999091

Heavens to Murgatroyd! These are practically the same sets!

• The mean and variance of X is the exact same across sets
• The mean and variance of Y is almost exactly the same across sets
• The correlation between X & Y is extremely close across sets
• The coefficient of determination is also extremely close

Once visualized, all four linear relationships appear to be exactly the same:

Well, that settles that. Pack it up, folks - the data are the same.

Hold up.

Wait a minute.

Something ain’t right.

Let’s try replotting the actual datasets and not just their linear models.

Boom! Not the same datasets at all.

Despite having the same mean, variance, correlations, and coefficients:

• Dataset 1 is a normally distributed linear relationship
• Dataset 2 is a parabolic curve
• Dataset 3 is a perfect linear relationship with a high-leverage outlier
• Dataset 4 shows absolutely no relationship but again has an outlier

Conclusion: Always conduct exploratory visualization as a staple of any analysis.

1.2.1 A Brief Overview

In 1999, statistician Leland Wilkinson published The Grammar of Graphics.

This framework allows us to dissect and alter plots in the same way we would a sentence.

Let’s begin with parts of viz, or layers.

1.2.2 Layers: Parts of Viz

In the grammar of graphics framework, each visualization is comprised of layers.

• Each layer performs a unique function in a visualization
• Like parts of speech, a layer can perform one function in infinite ways
  • For example, the data layer functions to input your data
  • A noun can be “fox” or “dog; a dataset can be”DOL" or “TSA”

1.2.3 Essential Layers: Data, Mappings, & Shapes

Like human sentences, every complete visualization has 3 essential layers:

Figure 1.2: The three essential layers for a complete visualization.

1.2.4 Remaining Layers

The 4 remaining layers in the grammar of graphics include:

• The coordinates layer modifies plot zoom, truncation, and labeling
• The statistics layer performs statistical transformations like lines of best fit
• The facets layer creates multiple comparison plots, a.k.a. small multiples
• The themes layer modifies font, gridlines, and other “non-data ink” polish

We’ll explore some key functions from these layers in the following sections.

1.5.1 The Aesthetics Layer

We’ve seen the principal function of the aesthetics layer, aes().

Most importantly, you should some arguments to which you can map variables:

• x = maps to the x-axis
• y = maps to the y-axis
• z = maps to the z-axis (3-dimensional, typically)
• color = maps to colors of points and lines
• fill = maps to colors of shapes, e.g. bars
• size = maps to the size of points
• labels = maps to the labels of points, lines, and shapes
• width = maps to the width of lines
• type = maps to the line type

Read more about aesthetics in the “ggplot2” vignette.

Take the following scatter plot of U.S. unemployed against total population since 1970:

library(ggplot2)

ggplot(data = economics, 
       aes(x = unemploy,
           y = pop)) +
  geom_point() +
  theme_minimal()

We can convey additional information by mapping aesthetics to other variables.

The following maps point size to psavert, or the personal savings rate.

library(ggplot2)

ggplot(data = economics, 
       aes(x = unemploy,
           y = pop,
           size = psavert)) +     # Map personal savings rate to size
  geom_point(alpha = 0.2) +       # Set transparency to 20%
  theme_minimal()

We can see that regardless of unemployment, savings dwindle as populations grow.

(Population, by the way, is practically interchangeable with date).

1.5.2 The Geometry Layer

Each shape your data takes in the geometry layer has an associated function.

• View all geometries on the “ggplot2” reference page
• Notably, every geometry function begins with geom_
• Some statistical functions have equivalents that begin with stat_
• You can combine more than one geom function in a singple visual

Rectangular Geoms: When using geoms with rectangular shapes, set stat = to “identity”.

options(scipen = 999)                       # Disable scientific notation

ggplot(data = hancock, 
       aes(x = reorder(ethnicity, -net),    # Reorder ethnicities by "net" 
           y = net)) +
  geom_bar(stat = "identity") +             # Specify stat = "identity"
  labs(x = NULL,
       y = "Weekly Net") +
  theme_minimal()

Use Fill to Modify Geoms: Map aes() argument fill = to a categorical variable:

ggplot(data = hancock, 
       aes(x = reorder(ethnicity, -net), 
           y = net,
           fill = reorder(class, -net))) +  # Add fill = and reorder() on "net"
  geom_bar(stat = "identity") +
  labs(x = NULL,
       y = "Weekly Net",
       fill = "Class") +                    # Use labs() and fill= to title legend
  theme_minimal()

Fill to 100%: With a fill = asthetic, set position = to “fill”:

ggplot(data = hancock, 
       aes(x = reorder(ethnicity, -net), 
           y = net,
           fill = reorder(class, -net))) +
  geom_bar(stat = "identity",
           position = "fill") +             # Set position = to "fill"
  labs(x = NULL,
       y = "Weekly Net",
       fill = "Class") +
  theme_minimal()

New Geoms with New Positions: With a fill = asthetic, set position = to “dodge”:

ggplot(data = hancock, 
       aes(x = reorder(ethnicity, -net), 
           y = net,
           fill = reorder(class, -net))) +
  geom_bar(stat = "identity",
           position = "dodge") +            # Set position = to "dodge"
  labs(x = NULL,
       y = "Weekly Net",
       fill = "Class") +
  theme_minimal()

Overlapping Barplots: Set position = to position_dodge() and set width =:

ggplot(data = hancock, 
       aes(x = reorder(ethnicity, -net), 
           y = net,
           fill = reorder(class, -net))) +
  geom_bar(stat = "identity",
           position = position_dodge(width = 0.5)) +            # Control the dodge!
  labs(x = NULL,
       y = "Weekly Net",
       fill = "Class") +
  theme_minimal()

Overlap & Transparency: When multiple points overlap, use transparency (alpha =).

ggplot(data = hancock, 
       aes(x = reorder(ethnicity, -net), 
           y = net)) +
  geom_point(alpha = 0.10) +          # Set transparency to 10%
  labs(x = NULL,
       y = "Weekly Net") +
  theme_minimal()

Overlapping & Jittering: In combination with transparency, you can “jitter” your points.

• “Jittering” adds random horizontal and vertical noise to break up points
• In function geom_point(), set position = to “jitter”
• Arguments height = and width = controls jitter
• Alternatively, use function geom_jitter()

ggplot(data = hancock, 
       aes(x = reorder(ethnicity, -net), 
           y = net)) +
  geom_jitter(alpha = 0.15,           # Change geom to geom_jitter()
              width = 0.25) +         # Set "jitter" width to 0.25
  labs(x = NULL,
       y = "Weekly Net") +
  theme_minimal()

1.5.3 The Statistics Layer

The statistics layer is where data undergo statistical transformations under the hood.

• All stat layer functions begin with stat_
• Many geometries already use stats, like geom_smooth()
• Argument method = can use different models with “lm”, “glm”, “rlm”, etc.
• Stat functions typically not called directly, but run within geom functions

Smoothers: Function stat_smooth() adds a LOESS line and standard error area.

ggplot(hancock, 
       aes(x = net,
           y = gross)) +
  geom_point() +
  stat_smooth() +                 # Function stat-smooth(), same as geom_smooth()
  theme_minimal()

Remove Standard Error: Set argument se = to FALSE to disable standard error.

ggplot(hancock, 
       aes(x = net,
           y = gross)) +
  geom_point() +
  stat_smooth(se = FALSE) +       # Argument se = controls standard error
  theme_minimal()

Chane Modeling Method: Set argument method = to “lm” for linear models, etc.

ggplot(hancock, 
       aes(x = net,
           y = gross)) +
  geom_point() +
  stat_smooth(se = FALSE,
              method = "lm") +    # Argument method = changes your model
  theme_minimal()

1.5.4 The Coordinates Layer: Planes

The coordinates layer ultimately controls the limits and labels of your axes.

• All coordinates functions begin with coord_
• Most common is coord_cartesian(), which modifies a plot’s Cartesian plane

Flipping Coordinates! No, we’re not cursing out coordinates.

However, you can reverse the x- and y-axes by using function coord_flip():

options(scipen = 999)                 # Disable scientific notation

ggplot(data = hancock, 
       aes(x = reorder(ethnicity, net), 
           y = net)) +
  geom_bar(stat = "identity") +
  labs(title = "Flipped!",
       x = NULL,
       y = "Weekly Net") +
  theme_minimal() +
  coord_flip()                        # coord_flip() !

Zooming or Filtering: Modifying coordinates typically goes one of two ways:

• Zoom in on some specified coordinates
• Zoom in and filter out data that are no longer visible

Zoom with coord_cartesian().

Filter with lims().

The Whole Plot (Before): Observe the entirety of the plot without zooming in:

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point() +
  geom_smooth() +
  theme_minimal()

Zoomed In: We can zoom, without affecting our data, with coordcartesion().

Note the direction and standard error of the LOESS line from geom_smooth().

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point() +
  geom_smooth() +
  theme_minimal() +
  coord_cartesian(xlim = c(0, 1250), 
                  ylim = c(0, 2000))    # Zoom in and keep your data!

Filtered: We can use lims() to zoom in and filter data no longer plotted.

Now note how the LOESS line and standard error have changed per this “new” dataset.

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point() +
  geom_smooth() +
  theme_minimal() +
  lims(x = c(0, 1250), 
       y = c(0, 2000))                  # Zoom in and filter your data!

1.5.5 The Coordinates Layer: Scales

Scales in “ggplot2” affect your axis gridlines, ticks, breaks, and labels.

• Scale functions begin with scale_
• Specify which axis with scale_x_ or scale_y_
• Also specify discrete, continuous, or another variable type
• Example: A discrete (categorical) variable on the y-axis is scale_y_discrete()

Axis Titles: Change (or remove) axis titles with argument name =:

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point(alpha = 0.05) +
  geom_smooth(color = "grey80", alpha = 0.1) +
  theme_minimal() +
  scale_x_continuous(name = "Weekly Net") +     # Axis? Continuous or discrete?
  scale_y_continuous(name = "Weekly Gross")     # Argument name = for axis titles

Breaks: Specify axes’ major label and tick marks with argument breaks = and c():

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point(alpha = 0.05) +
  geom_smooth(color = "grey80", alpha = 0.1) +
  theme_minimal() +
  scale_x_continuous(name = "Weekly Net",
                     breaks = c(0, 1250, 2500)) +
  scale_y_continuous(name = "Weekly Gross",
                     breaks = c(0, 1500, 3000))           # Even breaks

Labels: Customize individual labels with argument labels = (ojo - can get lengthy):

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point(alpha = 0.05) +
  geom_smooth(color = "grey80", alpha = 0.1) +
  theme_minimal() +
  scale_x_continuous(name = "Net",
                     breaks = c(0, 1250, 2500),
                     labels = c("$ 0", "1.25", "2.5 K")) +
  scale_y_continuous(name = "Gross",
                     breaks = c(0, 1500, 3000),
                     labels = c("$ 0", "1.5", "3 K"))     # Labels in quotes

Pretty Breaks: Package “scales” allows us to more easily format scale_*() functions.

• Format breaks automatically (pretty-like)
• Label dollar amounts and percentages with ease

Pretty Names: With “scales”, argument name = accepts new standards:

library(scales)

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point(alpha = 0.05) +
  geom_smooth(color = "grey80", alpha = 0.1) +
  theme_minimal() +
  scale_x_continuous(name = "Net",
                     breaks = c(0, 1250, 2500),
                     labels = dollar) +                   # Auto-labeling: Good
  scale_y_continuous(name = "Gross",
                     breaks = c(0, 1500, 3000),
                     labels = dollar)                     # Less flexibility: Bad

Pretty Breaks: With “scales”, argument breaks = accepts pretty_breaks():

library(scales)

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point(alpha = 0.05) +
  geom_smooth(color = "grey80", alpha = 0.1) +
  theme_minimal() +
  scale_x_continuous(name = "Net",
                     breaks = pretty_breaks(5),
                     labels = dollar) +                   # pretty_breaks() !
  scale_y_continuous(name = "Gross",
                     breaks = pretty_breaks(12),
                     labels = dollar)                     # Pretty sick

1.5.6 The Faceting Layer

The faceting layer allows us to create facets, a.k.a small multiples or trellis charts.

• All faceting functions begin with facet_
• facet_wrap() creates a grid; facet_grid() creates side-by-sides
• There are other arguments, but the formula x ~ y (variables), is critical

Single-Variable Faceting: The formula x ~ y needs only one side.

You can replace the empty side with . or leave it blank.

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point(alpha = 0.5) +
  geom_smooth(alpha = 0.5,
              method = "lm") +
  theme_minimal() +
  scale_x_continuous(name = "Net",
                     breaks = pretty_breaks(3),
                     labels = dollar) +
  scale_y_continuous(name = "Gross",
                     breaks = pretty_breaks(3),
                     labels = dollar) +
  facet_wrap(facets = ethnicity ~ .)                     # Only one side needed

Facet Grid: When you want side-by-side comparisons for all plots, use facet_grid():

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point(alpha = 0.5) +
  geom_smooth(alpha = 0.5,
              method = "lm") +
  theme_minimal() +
  scale_x_continuous(name = "Net",
                     breaks = pretty_breaks(3),
                     labels = dollar) +
  scale_y_continuous(name = "Gross",
                     breaks = pretty_breaks(3),
                     labels = dollar) +
  facet_grid(facets = . ~ ethnicity, )                     # Now with facet_grid() !

Modify facet_*() functions with arguments beyond the x ~ y formula.

• rows = accepts the number of rows in a facet grid
• cols = accepts the number of columns
• labeller = accepts new strip names

1.5.7 The Themes Layer

The themes layer allows you to customize ever pixel of non-data ink in your plot.

• Function themes() is the workhorse function for this layer
• themes() arguments are a large but consistent framework
• Functions beginning with theme_ set premade themes
• The themes layer adjusts every non-data pixel

Everything we’ve done so far, plus changing the facet strip labels:

ggplot(hancock,
       aes(x = net,
           y = gross)) +
  geom_point(alpha = 0.5,
             color = "grey30") +
  geom_smooth(alpha = 0.5,
              method = "lm") +
  theme_minimal() +
  scale_x_continuous(name = "Net",
                     breaks = pretty_breaks(3),
                     labels = dollar) +
  scale_y_continuous(name = "Gross",
                     breaks = pretty_breaks(3),
                     labels = dollar) +
  facet_wrap(facets = . ~ ethnicity) +
  theme(strip.text = element_text(face = "bold",
                                  color = "grey40",
                                  family = "Verdana"))   # The framework in action

So what the hell was that? There’s a whole consistent framework here!

There’s a few hierarchies of functions and arguments that we’ll look at below.

We’ve saved the above plot in object p, which we’ll use herein.

First Entering theme(): You’ll get hit with four short arguments:

1. line = modifies all lines in the plot
2. rect = modifies all rectangles in the plot
3. text = modifies all text in the plot
4. title = modifies just titles in the plot

p + theme(text = ...)

Latter Arguments in theme(): More specific argument families for each element.

• axis arguments for axis lines, ticks, and text
• legend arguments for boxes, keys, spacing, text, and titles
• panel arguments for overall plot spacing and grid lines
• plot arguments for tags, titles, and margins
• strip arguments for facet strip text and backgrounds

p + theme(axis.title = ...)

Once Argument is Selected: Arguments typically accept theme element functions:

• All element functions begin with element_
• element_rect() applies to rectangles
• element_line() applies to lines
• element_text() applies to text
• element_blank() removes it

p + theme(axis.title = element_text(color = "gray50",
                                    face = "bold",
                                    size = 10))