An Introduction to R

An Introduction to R Alex Douglas, Deon Roos, Francesca Mancini, Ana Couto & David Lusseau 2 An Introduction to R Alex Douglas, Deon Roos, Francesca Mancini, Ana Couto & David Lusseau April 9, 2024 2 Contents Preface 7 0.1 Theaimofthisbook............................ 7 0.2 Whoisthisbookfor?............................ 7 0.3 Whyanopenbook? ............................ 8 0.4 Whoarewe? ................................ 8 0.5 Howtousethisbook............................ 9 0.6 Bookwebsite ................................ 10 0.7 SomeRpointers .............................. 10 0.8 Thanks ................................... 11 0.9 License.................................... 11 1 Getting started with R and RStudio 13 1.1 InstallingR ................................. 14 1.2 InstallingRStudio ............................. 15 1.3 RStudioorientation............................. 17 1.4 Alternatives to RStudio . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.5 Rpackages ................................. 22 1.6 ProjectsinRStudio............................. 25 1.7 Workingdirectories............................. 29 1.8 Directorystructure............................. 31 1.9 Filenames.................................. 33 1.10 Project documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 1.11Rstyleguide ................................ 36 1.12Backingupprojects............................. 37 1.13CitingR................................... 38 1.14Exercise1.................................. 39 3 4CONTENTS 2 Some R basics 41 2.1 Gettingstarted ............................... 42 2.2 ObjectsinR................................. 43 2.3 UsingfunctionsinR ............................ 47 2.4 Workingwithvectors............................ 50 2.5 Gettinghelp................................. 57 2.6 SavingstuffinR .............................. 61 2.7 Exercise2.................................. 62 3 Data in R 63 3.1 Datatypes ................................. 63 3.2 Datastructures............................... 65 3.3 Importingdata ............................... 72 3.4 Wrangling data frames . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.5 Summarising data frames . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.6 Exportingdata ............................... 107 3.7 Exercise3.................................. 110 4 Graphics with base R 111 4.1 Gettingstarted ............................... 112 4.2 SimplebaseRplots............................. 113 4.3 Customisingplots.............................. 137 4.4 Multiplegraphs............................... 149 4.5 Exportingplots............................... 153 4.6 Exercise4.................................. 154 5 Graphics with ggplot 155 5.1 Beginningattheend............................ 156 5.2 Thestartoftheend ............................ 158 5.3 Tipsandtricks ............................... 193 5.4 Aggplotbestiary.............................. 216 5.5 Exercise5.................................. 228 CONTENTS 5 6 Simple Statistics in R 229 6.1 One and two sample tests . . . . . . . . . . . . . . . . . . . . . . . . . 229 6.2 Correlation ................................. 237 6.3 Simple linear modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 6.4 Other modelling approaches . . . . . . . . . . . . . . . . . . . . . . . . 253 6.5 Exercise6.................................. 254 7 Programming in R 255 7.1 Looking behind the curtain . . . . . . . . . . . . . . . . . . . . . . . . 255 7.2 FunctionsinR ............................... 256 7.3 Conditional statements . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 7.4 Combining logical operators . . . . . . . . . . . . . . . . . . . . . . . . 265 7.5 Loops .................................... 266 7.6 Exercise7.................................. 273 8 Reproducible reports with R markdown 275 8.1 WhatisRmarkdown?........................... 275 8.2 WhyuseRmarkdown?........................... 276 8.3 Get started with R markdown . . . . . . . . . . . . . . . . . . . . . . . 278 8.4 Create an R markdown document . . . . . . . . . . . . . . . . . . . . . 278 8.5 Rmarkdownanatomy ........................... 283 8.6 Sometipsandtricks ............................ 302 8.7 Further Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304 9 Version control with Git and GitHub 305 9.1 What is version control? . . . . . . . . . . . . . . . . . . . . . . . . . . 306 9.2 Why use version control? . . . . . . . . . . . . . . . . . . . . . . . . . . 306 9.3 What is Git and GitHub? . . . . . . . . . . . . . . . . . . . . . . . . . 307 9.4 Gettingstarted ............................... 308 9.5 Setting up a project in RStudio . . . . . . . . . . . . . . . . . . . . . . 311 9.6 UsingGit .................................. 322 9.7 Furtherresources.............................. 343 A Installing R Markdown 345 A.1 MSWindows ................................ 345 A.2 MacOSX .................................. 346 6CONTENTS Preface 0.1 The aim of this book The aim of this book is to introduce you to using R, a powerful and flexible interactive environment for statistical computing and research. R in itself is not difficult to learn, but as with learning any new language (spoken or computer) the initial learning curve can be a little steep and somewhat daunting. We have tried to simplify the content of this book as much as possible and have based it on our own personal experience of teaching (and learning) R over the last 15 years. It is not intended to cover everything there is to know about R - that would be an impossible task. Neither is it intended to be an introductory statistics course, although you will be using some simple statistics to highlight some of R’s capabilities. The main aim of this book is to help you climb the initial learning curve and provide you with the basic skills and experience (and confidence!) to enable you to further your experience in using R. 0.2 Who is this book for? We hope this book will be a useful introduction for anyone who wants to learn how to use R. It started out as a 100 page pdf manual used to support our teaching in numerous undergraduate and postgraduate biostatistics classes at the University of Aberdeen and has evolved over the last 10 years to the book you’re now reading. Consequently, this book is primarily written with the advanced undergraduate or postgraduate student in mind although the material should be useful for anyone with an interest in learning R regardless of background. Our book assumes that you have no previous experience of using either R or RStudio and no background in programming or using commandline driven software. We’ve also tried to make the content of this book operating system agnostic and have included information for Windows, Mac and Linux users where appropriate. 7 14 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO 1.1 Installing R To get up and running the first thing you need to do is install R. R is freely available for Windows, Mac and Linux operating systems from the Comprehensive R Archive Network (CRAN) website. For Windows and Mac users we suggest you download and install the pre-compiled binary versions. See this video for step-by-step instructions on how to download and install R and RStudio 1.1.1 Windows users For Windows users select the ‘Download R for Windows’ link and then click on the ‘base’ link and finally the download link ‘Download R 4.3.3 for Windows’. This will begin the download of the ‘.exe’ installation file. When the download has completed double click on the R executable file and follow the on-screen instructions. Full installation instructions can be found at the CRAN website. 1.1.2 Mac users For Mac users select the ‘Download R for (Mac) OS X’ link. The binary can be downloaded by selecting the ‘R-4.3.3.pkg’. Once downloaded, double click on the file icon and follow the on-screen instructions to guide you through the necessary steps. See the ‘R for Mac OS X FAQ’ for further information on installation. 1.1.3 Linux users For Linux users, the installation method will depend on which flavour of Linux you are using. There are reasonably comprehensive instruction here for Debian, Redhat, Suse and Ubuntu. In most cases you can just use your OS package manager to install R from the official repository. On Ubuntu fire up a shell (Terminal) and use (you will need root permission to do this): sudo apt update sudo apt install r-base r-base-dev which will install base R and also the development version of base R (you only need this if you want to compile R packages from source but it doesn’t hurt to have it). If you receive an error after running the code above you may need to add a ‘source.list’ entry to your etc/apt/sources.list file. To do this open the /etc/apt/sources.list file in your favourite text editor (gedit, vim, nano etc) and add the following line (you will need root permission to do this): 1.2. INSTALLING RSTUDIO 15 deb https://cloud.r-project.org/bin/linux/ubuntu disco-cran35/ This is the source.list for the latest version of Ubuntu (19.04 Disco Dingoat the time of writing). If you’re using an earlier version of Ubuntu then replace the source.list entry to the one which corresponds to the version of Ubuntu you are using (see here for an up to date list). Once you have done this then re-run the apt commands above and you should be good to go. 1.1.4 Testing R Whichever operating system you’re using, once you have installed R you need to check its working properly. The easiest way to do this is to start R by double clicking on the R icon (Windows or Mac) or by typing Rinto the Console (Linux). You should see the R Console and you should be able to type R commands into the Console after the command prompt >. Try typing the following R code and then press enter (don’t worry if you don’t understand this - we’re just checking if R works) plot(1:10) 2 4 6 8 10 2 4 6 8 10 Index 1:10 A plot of the numbers 1 to 10 on both the x and y axes should appear. If you see this, you’re good to go. If not then we suggest you make a note of any errors produced and then use Google to troubleshoot. 1.2 Installing RStudio Whilst its eminently possible to just use the base installation of R (many people do), we will be using a popular Integrated Development Environment (IDE) called RStudio. 16 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO RStudio can be thought of as an add-on to R which provides a more user-friendly interface, incorporating the R Console, a script editor and other useful functionality (like R markdown and GitHub integration). You can find more information about RStudio here. RStudio is freely available for Windows, Mac and Linux operating systems and can be downloaded from the RStudio site. You should select the ‘RStudio Desktop’ version. Note: you must install R before you install RStudio (see previous section for details). See this video for step-by-step instructions on how to download and install R and RStudio 1.2.1 Windows and Mac users For Windows and Mac users you should be presented with the appropriate link for downloading. Click on this link and once downloaded run the installer and follow the instructions. If you don’t see the link then scroll down to the ‘All Installers’ section and choose the link manually. 1.2.2 Linux users For Linux users scroll down to the ‘All Installers’ section and choose the appropriate link to download the binary for your Linux operating system. RStudio for Ubuntu (and Debian) is available as a *.deb package. The easiest way to install deb files on Ubuntu is by using the gdebi command. If gdebi is not available on your system you can install it by using the following command in the Terminal (you will need root permission to do this) sudo apt update sudo apt install gdebi-core To install the *.deb file navigate to where you downloaded the file and then enter the following command with root permission sudo gdebi rstudio-xenial-1.2.5XXX-amd64.deb where ‘-1.2.5XXX’ is the current version for Ubuntu (rstudio-xenial-1.2.5019-amd64.deb at the time of writing). You can then start RStudio from the Console by simply typing rstudio or you can create a shortcut on your Desktop for easy startup. 1.3. RSTUDIO ORIENTATION 17 1.2.3 Testing RStudio Once installed, you can check everything is working by starting up RStudio (you don’t need to start R as well, just RStudio). You should see something like the image below (if you’re on a Windows or Linux computer there may be small cosmetic differences). 1.3 RStudio orientation When you open R studio for the first time you should see the following layout (it might look slightly different on a Windows computer). The large window (aka pane) on the left is the Console window. The window on the top right is the Environment / History / Connections pane and the bottom 18 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO right window is the Files / Plots / Packages / Help / Viewer window. We will discuss each of these panes in turn below. You can customise the location of each pane by clicking on the ‘Tools’ menu then selecting Global Options –> Pane Layout. You can resize the panes by clicking and dragging the middle of the window borders in the direction you want. There are a plethora of other ways to customise RStudio. See this video for a quick introduction to RStudio 1.3.1 Console The Console is the workhorse of R. This is where R evaluates all the code you write. You can type R code directly into the Console at the command line prompt, >. For example, if you type 2+2into the Console you should obtain the answer 4(reassuringly). Don’t worry about the [1] at the start of the line for now. However, once you start writing more R code this becomes rather cumbersome. Instead of typing R code directly into the Console a better approach is to create an R script. An R script is just a plain text file with a .R file extension which contains your lines of R code. These lines of code are then sourced into the R Console line by line. To create a new R script click on the ‘File’ menu then select New File –> R Script. 1.3. RSTUDIO ORIENTATION 19 Notice that you have a new window (called the Source pane) in the top left of RStudio and the Console is now in the bottom left position. The new window is a script editor and where you will write your code. To source your code from your script editor to the Console simply place your cursor on the line of code and then click on the ‘Run’ button in the top right of the script editor pane. 20 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO You should see the result in the Console window. If clicking on the ‘Run’ button starts to become tiresome you can use the keyboard shortcut ‘ctrl + enter’ (on Windows and Linux) or ‘cmd + enter’ (on Mac). You can save your R scripts as a .R file by selecting the ‘File’ menu and clicking on save. Notice that the file name in the tab will turn red to remind you that you have unsaved changes. To open your R script in RStudio select the ‘File’ menu and then ‘Open File…’. Finally, its worth noting that although R scripts are saved with a .R extension they are actually just plain text files which can be opened with any text editor. 1.3.2 Environment/History/Connections The Environment / History / Connections window shows you lots of useful information. You can access each component by clicking on the appropriate tab in the pane. • The ‘Environment’ tab displays all the objects you have created in the current (global) environment. These objects can be things like data you have imported or functions you have written. Objects can be displayed as a List or in Grid format by selecting your choice from the drop down button on the top right of the window. If you’re in the Grid format you can remove objects from the environment by placing a tick in the empty box next to the object name and then click on the broom icon. There’s also an ‘Import Dataset’ button which will import data saved in a variety of file formats. However, we would suggest that you don’t use this approach to import your data as it’s not reproducible and therefore not robust (see Chapter 3 for more details). • The ‘History’ tab contains a list of all the commands you have entered into the R Console. You can search back through your history for the line of code you have forgotten, send selected code back to the Console or Source window. We usually never use this as we always refer back to our R script. • The ‘Connections’ tab allows you to connect to various data sources such as external databases. 1.4. ALTERNATIVES TO RSTUDIO 21 1.3.3 Files/Plots/Packages/Help/Viewer • The ‘Files’ tab lists all external files and directories in the current working directory on your computer. It works like file explorer (Windows) or Finder (Mac). You can open, copy, rename, move and delete files listed in the window. • The ‘Plots’ tab is where all the plots you create in R are displayed (unless you tell R otherwise). You can ‘zoom’ into the plots to make them larger using the magnifying glass button, and scroll back through previously created plots using the arrow buttons. There is also the option of exporting plots to an external file using the ‘Export’ drop down menu. Plots can be exported in various file formats such as jpeg, png, pdf, tiff or copied to the clipboard (although you are probably better off using the appropriate R functions to do this - see Chapter 4 for more details). • The ‘Packages’ tab lists all of the packages that you have installed on your computer. You can also install new packages and update existing packages by clicking on the ‘Install’ and ‘Update’ buttons respectively. • The ‘Help’ tab displays the R help documentation for any function. We will go over how to view the help files and how to search for help in Chapter 2. • The ‘Viewer’ tab displays local web content such as web graphics generated by some packages. 1.4 Alternatives to RStudio Although RStudio is becoming increasingly popular it might not be the best choice for everyone and you certainly don’t have to use it to use R effectively. Rather than using an ‘all in one’ IDE many people choose to use R and a separate script editor to write and execute R code. If you’re not familiar with what a script editor is, you can think of it as a bit like a word processor but specifically designed for writing code. Happily, there are many script editors freely available so feel free to download and experiment until you find one you like. Some script editors are only available for certain operating systems and not all are specific to R. Suggestions for script editors are provided below. Which one you choose is up to you: one of the great things about R is that YOU get to choose how you want to use R. 1.4.1 Advanced text editors A light yet efficient way to write your R scripts using advanced text editors such as: •Atom (all operating systems) •BBedit (Mac OS) •gedit (Linux; comes with most Linux distributions) •MacVim (Mac OS) 22 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO •Nano (Linux) •Notepad++ (Windows) •Sublime Text (all operating systems) 1.4.2 Integrated development environments These environments are more powerful than simple text editors, and are similar to RStudio: •Emacs and its extension Emacs Speaks Statistics (all operating systems) •RKWard (Linux) •Tinn-R (Windows) •vim and its extension NVim-R (Linux) 1.5 R packages The base installation of R comes with many useful packages as standard. These packages will contain many of the functions you will use on a daily basis. However, as you start using R for more diverse projects (and as your own use of R evolves) you will find that there comes a time when you will need to extend R’s capabilities. Happily, many thousands of R users have developed useful code and shared this code as installable packages. You can think of a package as a collection of functions, data and help files collated into a well defined standard structure which you can download and install in R. These packages can be downloaded from a variety of sources but the most popular are CRAN,Bioconductor and GitHub. Currently, CRAN hosts over 15000 packages and is the official repository for user contributed R packages. Bioconductor provides open source software oriented towards bioinformatics and hosts over 1800 R packages. GitHub is a website that hosts git repositories for all sorts of software and projects (not just R). Often, cutting edge development versions of R packages are hosted on GitHub so if you need all the new bells and whistles then this may be an option. However, a potential downside of using the development version of an R package is that it might not be as stable as the version hosted on CRAN (it’s in development!) and updating packages won’t be automatic. 1.5.1 CRAN packages See this video for step-by-step instruction on how to install, use and update packages from CRAN 1.5. R PACKAGES 23 To install a package from CRAN you can use the install.packages() function. For example if you want to install the remotes package enter the following code into the Console window of RStudio (note: you will need a working internet connection to do this) install.packages('remotes',dependencies = TRUE) You may be asked to select a CRAN mirror, just select ‘0-cloud’ or a mirror near to your location. The dependencies = TRUE argument ensures that additional packages that are required will also be installed. It’s good practice to occasionally update your previously installed packages to get access to new functionality and bug fixes. To update CRAN packages you can use the update.packages() function (you will need a working internet connection for this). update.packages(ask = FALSE) The ask = FALSE argument avoids having to confirm every package download which can be a pain if you have many packages installed. 1.5.2 Bioconductor packages To install packages from Bioconductor the process is a little different. You first need to install the BiocManager package. You only need to do this once unless you subsequently reinstall or upgrade R. install.packages('BiocManager',dependencies = TRUE) Once the BiocManager package has been installed you can either install all of the ‘core’ Bioconductor packages with BiocManager::install() or install specific packages such as the ‘GenomicRanges’ and ‘edgeR’ packages. BiocManager::install(c("GenomicRanges","edgeR")) To update Bioconductor packages just use the BiocManager::install() function again. 30 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO In the example above, the working directory is a folder called ‘first_project’ which is a subfolder of “Teaching’ in the ‘Alex’ folder which in turn is in a ‘Documents’ folder located in the ‘nhy163’ folder which itself is in the ‘Users’ folder. On a Windows based computer our working directory would also include a drive letter (i.e. C:/Users/nhy163/Documents/Alex/Teaching/first_project). If you weren’t using an RStudio Project then you would have to set your working directory using the setwd() function at the start of every R script (something we did for many years). setwd('/Users/nhy163/Documents/Alex/Teaching/first_project') However, the problem with setwd() is that it uses an absolute file path which is specific to the computer you are working on. If you want to send your script to someone else (or if you’re working on a different computer) this absolute file path is not going to work on your friend/colleagues computer as their directory configuration will be different (you are unlikely to have a directory structure /Users/nhy163/Documents/Alex/Teaching/ on your computer). This results in a project that is not self-contained and not easily portable. RStudio solves this problem by allowing you to use relative file paths which are relative to the Root project directory. The Root project directory is just the directory that contains the .Rproj file (first_project.Rproj in our case). If you want to share your analysis with someone else, all you need to do is copy the entire project directory and send to your to your collaborator. They would then just need to open the project file and any R scripts that contain references to relative file paths will just work. For example, let’s say that you’ve created a subdirectory called raw_data in your Root project directory that contains a tab delimited datafile called mydata.txt (we will cover directory structures below). To import this datafile in an RStudio project using the read.table() function (don’t worry about this now, we will cover this in much more detail in Chapter 3) all you need to include in your R script is 1.8. DIRECTORY STRUCTURE 31 dataf <- read.table('raw_data/mydata.txt',header = TRUE, sep = '\t') Because the file path raw_data/mydata.txt is relative to the project directory it doesn’t matter where you collaborator saves the project directory on their computer it will still work. If you weren’t using an RStudio project then you would have to use either of the options below neither of which would work on a different computer. setwd("/Users/nhy163/Documents/Alex/Teaching/first_project/") dataf <- read.table("raw_data/mydata.txt",header = TRUE,sep = "\t") # or dataf <- read.table("/Users/nhy163/Documents/Alex/Teaching/first_project/raw_data/mydata.txt", header = TRUE,sep = "\t") For those of you who want to take the notion of relative file paths a step further, take a look at the here() function in the here package. The here() function allows you to automagically build file paths for any file relative to the project root directory that are also operating system agnostic (works on a Mac, Windows or Linux machine). For example, to import our mydata.txt file from the raw_data directory just use library(here) # you may need to install the here package first dataf <- read.table(here("raw_data","mydata.txt"), header = TRUE,sep = '\t', stringsAsFactors = TRUE) # or without loading the here package dataf <- read.table(here::here("raw_data","mydata.txt"), header = TRUE,sep = '\t', stringsAsFactors = TRUE) 1.8 Directory structure In addition to using RStudio Projects, it’s also really good practice to structure your working directory in a consistent and logical way to help both you and your collaborators. We frequently use the following directory structure in our R based projects 32 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO In our working directory we have the following directories: •Root - This is your project directory containing your .Rproj file. •data - We store all our data in this directory. The subdirectory called raw_data contains raw data files and only raw data files. These files should be treated as read only and should not be changed in any way. If you need to process/clean/modify your data do this in R (not MS Excel) as you can document (and justify) any changes made. Any processed data should be saved to a separate file and stored in the processed_data subdirectory. Information about data collection methods, details of data download and any other useful metadata should be saved in a text document (see README text files below) in the metadata subdirectory. •RThis is an optional directory where we save all of the custom R functions we’ve written for the current analysis. These can then be sourced into R using the source() function. •Rmd - An optional directory where we save our R markdown documents. •scripts - All of the main R scripts we have written for the current project are saved here. •output - Outputs from our R scripts such as plots, HTML files and data summaries are saved in this directory. This helps us and our collaborators distinguish what files are outputs and which are source files. Of course, the structure described above is just what works for us most of the time and should be viewed as a starting point for your own needs. We tend to have a fairly consistent directory structure across our projects as this allows us to quickly orientate ourselves when we return to a project after a while. Having said that, different projects will have different requirements so we happily add and remove directories as required. 1.9. FILE NAMES 33 You can create your directory structure using Windows Explorer (or Finder on a Mac) or within RStudio by clicking on the ‘New folder’ button in the ‘Files’ pane. An alternative approach is to use the dir.create() and list.files() functions in the R Console. # create directory called 'data' dir.create('data') # create subdirectory raw_data in the data directory dir.create('data/raw_data') # list the files and directories list.files(recursive = TRUE,include.dirs = TRUE) # [1] "data" "data/raw_data" "first_project.Rproj" 1.9 File names What you call your files matters more than you might think. Naming files is also more difficult than you think. The key requirement for a ‘good’ file name is that it’s informative whilst also being relatively short. This is not always an easy compromise and often requires some thought. Ideally you should try to avoid the following! Although there’s not really a recognised standard approach to naming files (actually there is, just not everyone uses it), there are a couple of things to bear in mind. • First, avoid using spaces in file names by replacing them with underscores or even hyphens. Why does this matter? One reason is that some command line software (especially many bioinformatic tools) won’t recognise a file name with a space 34 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO Figure 1.1: source:https://xkcd.com/1459/ and you’ll have to go through all sorts of shenanigans using escape characters to make sure spaces are handled correctly. Even if you don’t think you will ever use command line software you may be doing so indirectly. Take R markdown for example, if you want to render an R markdown document to pdf using the rmarkdown package you will actually be using a command line LaTeX engine under the hood (called Pandoc). Another good reason not to use spaces in file names is that it makes searching for file names (or parts of file names) using regular expressions in R (or any other language) much more difficult. • For the reasons given above, also avoid using special characters (i.e. @£$%^&*(:/) in your file names. • If you are versioning your files with sequential numbers (i.e. file1, file2, file3 …) and you have more than 9 files you should use 01, 02, 03 .. 10 as this will ensure the files are printed in the correct order (see what happens if you don’t). If you have more than 99 files then use 001, 002, 003… etc. • If your file names include dates, use the ISO 8601 format YYYY-MM-DD (or YYYYMMDD) to ensure your files are listed in proper chronological order. • Never use the word final in any file name - it never is! Whatever file naming convention you decide to use, try to adopt early, stick with it and be consistent. You’ll thank us! 1.10 Project documentation A quick note or two about writing R code and creating R scripts. Unless you’re doing something really quick and dirty we suggest that you always write your R code as an 1.10. PROJECT DOCUMENTATION 35 R script. R scripts are what make R so useful. Not only do you have a complete record of your analysis, from data manipulation, visualisation and statistical analysis, you can also share this code (and data) with friends, colleagues and importantly when you submit and publish your research to a journal. With this in mind, make sure you include in your R script all the information required to make your work reproducible (author names, dates, sampling design etc). This information could be included as a series of comments #or, even better, by mixing executable code with narrative into an R markdown document. It’s also good practice to include the output of the sessionInfo() function at the end of any script which prints the R version, details of the operating system and also loaded packages. A really good alternative is to use the session_info() function from the xfun package for a more concise summary of our session environment. Here’s an example of including meta-information at the start of an R script. # Title: Time series analysis of snouters # Purpose : This script performs a time series analyses on # snouter count data. # Data consists of counts of snouter species # collected from 18 islands in the Hy-yi-yi # archipelago between 1950 and 1957. # For details of snouter biology see: # https://en.wikipedia.org/wiki/Rhinogradentia # Project number: #007 # DataFile:'data/snouter_pop.txt' # Author: A. Nother # Contact details: [email protected] # Date script created: Mon Dec 2 16:06:44 2019 ----------- # Date script last modified: Thu Dec 12 16:07:12 2019 ---- # package dependencies library(PopSnouter) library(ggplot2) print('put your lovely R code here') # good practice to include session information xfun::session_info() This is just one example and there are no hard and fast rules so feel free to develop a system that works for you. A really useful shortcut in RStudio is to automatically 36 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO include a time and date stamp in your R script. To do this, write ts where you want to insert your time stamp in your R script and then press the ‘shift + tab’ keys. RStudio will magically convert ts into the current date and time and also automatically comment out this line with a #. Another really useful RStudio shortcut is to comment out multiple lines in your script with a #symbol. To do this, highlight the lines of text you want to comment and then press ‘ctrl + shift + c’ (or ‘cmd + shift + c’ on a mac). To uncomment the lines just use ‘ctrl + shift + c’ again. In addition to including metadata in your R scripts it’s also common practice to create a separate text file to record important information. By convention these text files are named README. We often include a README file in the directory where we keep our raw data. In this file we include details about when data were collected (or downloaded), how data were collected, information about specialised equipment, preservation methods, type and version of any machines used (i.e. sequencing equipment) etc. You can create a README file for your project in RStudio by clicking on the File -> New File -> Text File menu. 1.11 R style guide How you write your code is more or less up to you although your goal should be to make it as easy to read as possible (for you and others). Whilst there are no rules (and no code police), we encourage you to get into the habit of writing readable R code by adopting a particular style. We suggest that you follow Google’s R style guide whenever possible. This style guide will help you decide where to use spaces, how to indent code and how to use square [ ] and curly { } brackets amongst other things. If all that sounds like too much hard work you can install the styler package which includes an RStudio add-in to allow you to automatically restyle selected code (or entire files and projects) with the click of your mouse. You can find more information about the styler package including how to install here. Once installed, you can highlight the code you want to restyle, click on the ‘Addins’ button at the top of RStudio and select the ‘Style Selection’ option. Here is an example of poorly formatted R code. Now highlight the code and use the styler package to reformat 1.12. BACKING UP PROJECTS 37 To produce some nicely formatted code 1.12 Backing up projects Don’t be that person who loses hard won (and often expensive) data and analyses. Don’t be that person who thinks it’ll never happen to me - it will! Always think of the absolute worst case scenario, something that makes you wake up in a cold sweat at night, and do all you can to make sure this never happens. Just to be clear, if you’re relying on copying your precious files to an external hard disk or USB stick this is NOT an effective backup strategy. These things go wrong all the time as you lob them into your rucksack or ‘bag for life’ and then lug them between your office and home. Even if you do leave them plugged into your computer what happens when the building burns down (we did say worst case!)? Ideally, your backups should be offsite and incremental. Happily there are numerous options for backing up your files. The first place to look is in your own institute. Most (all?) Universities have some form of network based storage that should be easily accessible and is also underpinned by a comprehensive disaster recovery plan. Other options include cloud based services such as Google Drive and Dropbox (to 38 CHAPTER 1. GETTING STARTED WITH R AND RSTUDIO name but a few), but make sure you’re not storing sensitive data on these services and are comfortable with the often eye watering privacy policies. Whilst these services are pretty good at storing files, they don’t really help with incremental backups. Finding previous versions of files often involves spending inordinate amounts of time trawling through multiple files named ‘final.doc’,‘final_v2.doc’ and ‘final_usethisone.doc’ etc until you find the one you were looking for. The best way we know for both backing up files and managing different versions of files is to use Git and GitHub. To find out more about how you can use RStudio, Git and GitHub together see the Git and GitHub Chapter. 1.13 Citing R Many people have invested huge amounts of time and energy making R the great piece of software you’re now using. If you use R in your work (and we hope you do) please remember to give appropriate credit by citing R. To get the most up to date citation for R you can use the citation() function. citation() ## To cite R in publications use: ## ## R Core Team (2023). _R: A Language and Environment for Statistical ## Computing_. R Foundation for Statistical Computing, Vienna, Austria. ## <https://www.R-project.org/>. ## ## A BibTeX entry for LaTeX users is ## ## @Manual{, ## title = {R: A Language and Environment for Statistical Computing}, ## author = {{R Core Team}}, ## organization = {R Foundation for Statistical Computing}, ## address = {Vienna, Austria}, ## year = {2023}, ## url = {https://www.R-project.org/}, ## } ## ## We have invested a lot of time and effort in creating R, please cite it ## when using it for data analysis. See also 'citation("pkgname")' for ## citing R packages. If you want to cite a particular package you’ve used for your data analysis. 1.14. EXERCISE 1 39 citation(package = "here") ## To cite package 'here' in publications use: ## ## Müller K (2020). _here: A Simpler Way to Find Your Files_. R package ## version 1.0.1, <https://CRAN.R-project.org/package=here>. ## ## A BibTeX entry for LaTeX users is ## ## @Manual{, ## title = {here: A Simpler Way to Find Your Files}, ## author = {Kirill Müller}, ## year = {2020}, ## note = {R package version 1.0.1}, ## url = {https://CRAN.R-project.org/package=here}, ## } 1.14 Exercise 1 Congratulations, you’ve reached the end of Chapter 1! Perhaps now’s a good time to practice some of what you’ve learned. You can find an exercise we’ve prepared for you (and our solutions) on the course website. 46 CHAPTER 2. SOME R BASICS char_obj <- "hello" char_obj2 <- "world!" char_obj3 <- char_obj +char_obj2 Error in char_obj+char_obj2:non-numeric argument to binary operator The error message is essentially telling you that either one or both of the objects char_obj and char_obj2 is not a number and therefore cannot be added together. When you first start learning R, dealing with errors and warnings can be frustrating as they’re often difficult to understand (what’s an argument? what’s a binary operator?). One way to find out more information about a particular error is to Google a generalised version of the error message. For the above error try Googling ‘non-numeric argument to binary operator error + r’ or even ‘common r error messages’. Another error message that you’ll get quite a lot when you first start using R is Error: object 'XXX' not found. As an example, take a look at the code below. my_obj <- 48 my_obj4 <- my_obj +no_obj Error:object 'no_obj' not found R returns an error message because we haven’t created (defined) the object no_obj yet. Another clue that there’s a problem with this code is that, if you check your environment, you’ll see that object my_obj4 has not been created. 2.2.2 Naming objects Naming your objects is one of the most difficult things you will do in R (honestly - we’re serious). Ideally your object names should be kept both short and informative which is not always easy. If you need to create objects with multiple words in their name then use either an underscore or a dot between words or capitalise the different words. We prefer the underscore format (called snake case). output_summary <- "my analysis" output.summary <- "my analysis" outputSummary <- "my analysis" There are also a few limitations when it come to giving objects names. An object name cannot start with a number or a dot followed by a number (i.e. 2my_variable or .2my_variable). You should also avoid using non-alphanumeric characters in your object names (i.e. &, ^, /, ! etc). In addition, make sure you don’t name your objects with reserved words (i.e. TRUE,NA) and it’s never a good idea to give your object the same name as a built-in function. One that crops up more times than we can remember is 2.3. USING FUNCTIONS IN R 47 data <- read.table("mydatafile",header = TRUE)#data is a # function! 2.3 Using functions in R Up until now we’ve been creating simple objects by directly assigning a single value to an object. It’s very likely that you’ll soon want to progress to creating more complicated objects as your R experience grows and the complexity of your tasks increase. Happily, R has a multitude of functions to help you do this. You can think of a function as an object which contains a series of instructions to perform a specific task. The base installation of R comes with many functions already defined or you can increase the power of R by installing one of the 10000’s of packages now available. Once you get a bit more experience with using R you may want to define your own functions to perform tasks that are specific to your goals (more about this in Chapter 7). See this video for a general introduction to using functions in R and this video on how to create vectors in R The first function we will learn about is the c() function. The c() function is short for concatenate and we use it to join together a series of values and store them in a data structure called a vector (more on vectors in Chapter 3). my_vec <- c(2,3,1,6,4,3,3,7) In the code above we’ve created an object called my_vec and assigned it a value using the function c(). There are a couple of really important points to note here. Firstly, when you use a function in R, the function name is always followed by a pair of round brackets even if there’s nothing contained between the brackets. Secondly, the argument(s) of a function are placed inside the round brackets and are separated by commas. You can think of an argument as way of customising the use or behaviour of a function. In the example above, the arguments are the numbers we want to concatenate. Finally, one of the tricky things when you first start using R is to know which function to use for a particular task and how to use it. Thankfully each function will always have a help document associated with it which will explain how to use the function (more on this later) and a quick Google search will also usually help you out. To examine the value of our new object we can simply type out the name of the object as we did before. 48 CHAPTER 2. SOME R BASICS my_vec ##[1]23164337 Now that we’ve created a vector we can use other functions to do useful stuff with this object. For example, we can calculate the mean, variance, standard deviation and number of elements in our vector by using the mean(),var(),sd() and length() functions. mean(my_vec) # returns the mean of my_vec ## [1] 3.625 var(my_vec) # returns the variance of my_vec ## [1] 3.982143 sd(my_vec) # returns the standard deviation of my_vec ## [1] 1.995531 length(my_vec) # returns the number of elements in my_vec ## [1] 8 If we wanted to use any of these values later on in our analysis we can just assign the resulting value to another object. vec_mean <- mean(my_vec) # returns the mean of my_vec vec_mean ## [1] 3.625 Sometimes it can be useful to create a vector that contains a regular sequence of values in steps of one. Here we can make use of a shortcut using the :symbol. my_seq <- 1:10 # create regular sequence my_seq ##[1]12345678910 my_seq2 <- 10:1# in decending order my_seq2 ##[1]10987654321 Other useful functions for generating vectors of sequences include the seq() and rep() functions. For example, to generate a sequence from 1 to 5 in steps of 0.5. my_seq2 <- seq(from = 1,to = 5,by = 0.5) my_seq2 ## [1] 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.3. USING FUNCTIONS IN R 49 Here we’ve used the arguments from = and to = to define the limits of the sequence and the by = argument to specify the increment of the sequence. Play around with other values for these arguments to see their effect. The rep() function allows you to replicate (repeat) values a specified number of times. To repeat the value 2, 10 times my_seq3 <- rep(2,times = 10)# repeats 2, 10 times my_seq3 ## [1]2222222222 You can also repeat non-numeric values my_seq4 <- rep("abc",times = 3)# repeats ‘abc’ 3 times my_seq4 ## [1] "abc" "abc" "abc" or each element of a series my_seq5 <- rep(1:5,times = 3)# repeats the series 1 to # 5, 3 times my_seq5 ## [1]123451234512345 or elements of a series. my_seq6 <- rep(1:5,each = 3)# repeats each element of the #series 3 times my_seq6 ## [1]111222333444555 We can also repeat a non-sequential series. my_seq7 <- rep(c(3,1,10,7), each = 3)# repeats each # element of the # series 3 times my_seq7 ## [1] 3 3 3 1 1 1 10 10 10 7 7 7 50 CHAPTER 2. SOME R BASICS Note in the code above how we’ve used the c() function inside the rep() function. Nesting functions allows us to build quite complex commands within a single line of code and is a very common practice when using R. However, care needs to be taken as too many nested functions can make your code quite difficult for others to understand (or yourself some time in the future!). We could rewrite the code above to explicitly separate the two different steps to generate our vector. Either approach will give the same result, you just need to use your own judgement as to which is more readable. in_vec <- c(3,1,10,7) my_seq7 <- rep(in_vec, each = 3)# repeats each element of # the series 3 times my_seq7 ## [1] 3 3 3 1 1 1 10 10 10 7 7 7 2.4 Working with vectors Manipulating, summarising and sorting data using R is an important skill to master but one which many people find a little confusing at first. We’ll go through a few simple examples here using vectors to illustrate some important concepts but will build on this in much more detail in Chapter 3 where we will look at more complicated (and useful) data structures. Take a look at this video for a quick introduction to working with vectors in R using positional and logical indexes 2.4.1 Extracting elements To extract (also known as indexing or subscripting) one or more values (more generally known as elements) from a vector we use the square bracket [ ] notation. The general approach is to name the object you wish to extract from, then a set of square brackets with an index of the element you wish to extract contained within the square brackets. This index can be a position or the result of a logical test. Positional index To extract elements based on their position we simply write the position inside the [ ]. For example, to extract the 3rd value of my_vec 2.4. WORKING WITH VECTORS 51 my_vec # remind ourselves what my_vec looks like ##[1]23164337 my_vec[3]# extract the 3rd value ## [1] 1 # if you want to store this value in another object val_3 <- my_vec[3] val_3 ## [1] 1 Note that the positional index starts at 1 rather than 0 like some other programming languages (i.e. Python). We can also extract more than one value by using the c() function inside the square brackets. Here we extract the 1st, 5th, 6th and 8th element from the my_vec object my_vec[c(1,5,6,8)] ##[1]2437 Or we can extract a range of values using the :notation. To extract the values from the 3rd to the 8th elements. my_vec[3:8] ##[1]164337 Logical index Another really useful way to extract data from a vector is to use a logical expression as an index. For example, to extract all elements with a value greater than 4 in the vector my_vec my_vec[my_vec >4] ## [1] 6 7 Here, the logical expression is my_vec > 4 and R will only extract those elements that satisfy this logical condition. So how does this actually work? If we look at the output of just the logical expression without the square brackets you can see that R returns a vector containing either TRUE or FALSE which correspond to whether the logical condition is satisfied for each element. In this case only the 4th and 8th elements return a TRUE as their value is greater than 4. 52 CHAPTER 2. SOME R BASICS my_vec >4 ## [1] FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE So what R is actually doing under the hood is equivalent to my_vec[c(FALSE,FALSE,FALSE,TRUE,FALSE,FALSE,FALSE,TRUE)] ## [1] 6 7 and only those element that are TRUE will be extracted. In addition to the <and >operators you can also use composite operators to increase the complexity of your expressions. For example the expression for ‘greater or equal to’ is >=. To test whether a value is equal to a value we need to use a double equals symbol == and for ‘not equal to’ we use != (the !symbol means ‘not’). my_vec[my_vec >= 4]# values greater or equal to 4 ## [1] 6 4 7 my_vec[my_vec <4]# values less than 4 ##[1]23133 my_vec[my_vec <= 4]# values less than or equal to 4 ##[1]231433 my_vec[my_vec == 4]# values equal to 4 ## [1] 4 my_vec[my_vec != 4]# values not equal to 4 ##[1]2316337 We can also combine multiple logical expressions using Boolean expressions. In R the &symbol means AND and the |symbol means OR. For example, to extract values in my_vec which are less than 6 AND greater than 2 val26 <- my_vec[my_vec <6&my_vec >2] val26 ##[1]3433 or extract values in my_vec that are greater than 6 OR less than 3. val63 <- my_vec[my_vec >6|my_vec <3] val63 ## [1] 2 1 7 2.4. WORKING WITH VECTORS 53 2.4.2 Replacing elements We can change the values of some elements in a vector using our [ ] notation in combination with the assignment operator <-. For example, to replace the 4th value of our my_vec object from 6to 500 my_vec[4]<- 500 my_vec ## [1] 2 3 1 500 4 3 3 7 We can also replace more than one value or even replace values based on a logical expression. # replace the 6th and 7th element with 100 my_vec[c(6,7)] <- 100 my_vec ## [1] 2 3 1 500 4 100 100 7 # replace element that are less than or equal to 4 with 1000 my_vec[my_vec <= 4]<- 1000 my_vec ## [1] 1000 1000 1000 500 1000 100 100 7 2.4.3 Ordering elements In addition to extracting particular elements from a vector we can also order the values contained in a vector. To sort the values from lowest to highest value we can use the sort() function. vec_sort <- sort(my_vec) vec_sort ## [1] 7 100 100 500 1000 1000 1000 1000 To reverse the sort, from highest to lowest, we can either include the decreasing = TRUE argument when using the sort() function vec_sort2 <- sort(my_vec, decreasing = TRUE) vec_sort2 ## [1] 1000 1000 1000 1000 500 100 100 7 54 CHAPTER 2. SOME R BASICS or first sort the vector using the sort() function and then reverse the sorted vector using the rev() function. This is another example of nesting one function inside another function. vec_sort3 <- rev(sort(my_vec)) vec_sort3 ## [1] 1000 1000 1000 1000 500 100 100 7 Whilst sorting a single vector is fun, perhaps a more useful task would be to sort one vector according to the values of another vector. To do this we should use the order() function in combination with [ ]. To demonstrate this let’s create a vector called height containing the height of 5 different people and another vector called p.names containing the names of these people (so Joanna is 180 cm, Charlotte is 155 cm etc). height <- c(180,155,160,167,181) height ## [1] 180 155 160 167 181 p.names <- c("Joanna","Charlotte","Helen","Karen","Amy") p.names ## [1] "Joanna" "Charlotte" "Helen" "Karen" "Amy" Our goal is to order the people in p.names in ascending order of their height. The first thing we’ll do is use the order() function with the height variable to create a vector called height_ord. height_ord <- order(height) height_ord ##[1]23415 OK, what’s going on here? The first value, 2, (remember ignore [1]) should be read as ‘the smallest value of height is the second element of the height vector’. If we check this by looking at the height vector above, you can see that element 2 has a value of 155, which is the smallest value. The second smallest value in height is the 3rd element of height, which when we check is 160 and so on. The largest value of height is element 5 which is 181. Now that we have a vector of the positional indices of heights in ascending order (height_ord), we can extract these values from our p.names vector in this order. names_ord <- p.names[height_ord] names_ord ## [1] "Charlotte" "Helen" "Karen" "Joanna" "Amy" 2.4. WORKING WITH VECTORS 55 You’re probably thinking ‘what’s the use of this?’ Well, imagine you have a dataset which contains two columns of data and you want to sort each column. If you just use sort() to sort each column separately, the values of each column will become uncoupled from each other. By using the order() on one column, a vector of positional indices is created of the values of the column in ascending order. This vector can be used on the second column, as the index of elements which will return a vector of values based on the first column. 2.4.4 Vectorisation One of the great things about R functions is that most of them are vectorised. This means that the function will operate on all elements of a vector without needing to apply the function on each element separately. For example, to multiple each element of a vector by 5 we can simply use # create a vector my_vec2 <- c(3,5,7,1,9,20) # multiply each element by 5 my_vec2 *5 ## [1] 15 25 35 5 45 100 Or we can add the elements of two or more vectors # create a second vector my_vec3 <- c(17,15,13,19,11,0) # add both vectors my_vec2 +my_vec3 ## [1] 20 20 20 20 20 20 # multiply both vectors my_vec2 *my_vec3 ## [1] 51 75 91 19 99 0 However, you must be careful when using vectorisation with vectors of different lengths as R will quietly recycle the elements in the shorter vector rather than throw a wobbly (error). # create a third vector my_vec4 <- c(1,2) 62 CHAPTER 2. SOME R BASICS save(nameOfObject, file = "name_of_file.RData") or if you want to save all of the objects in your workspace into a single .RData file use the save.image() function. save.image(file = "name_of_file.RData") To load your .RData file back into RStudio use the load() function. load(file = "name_of_file.RData") 2.7 Exercise 2 Congratulations, you’ve reached the end of Chapter 2! Perhaps now’s a good time to practice some of what you’ve learned. You can find an exercise we’ve prepared for you (and our solutions) on the course website. Chapter 3 Data in R Until now, you’ve created fairly simple data in R and stored it as a vector. However, most (if not all) of you will have much more complicated datasets from your various experiments and surveys that go well beyond what a vector can handle. Learning how R deals with different types of data and data structures, how to import your data into R and how to manipulate and summarise your data are some of the most important skills you will need to master. In this Chapter we’ll go over the main data types in R and focus on some of the most common data structures. We will also cover how to import data into R from an external file, how to manipulate (wrangle) and summarise data and finally how to export data from R to an external file. 3.1 Data types Understanding the different types of data and how R deals with these data is important. The temptation is to glaze over and skip these technical details, but beware, this can come back to bite you somewhere unpleasant if you don’t pay attention. We’ve already seen an example of this when we tried (and failed) to add two character objects together using the +operator. R has six basic types of data; numeric, integer, logical, complex and character. The keen eyed among you will notice we’ve only listed five data types here, the final data type is raw which we won’t cover as it’s not useful 99.99% of the time. We also won’t cover complex numbers as we don’t have the imagination! •Numeric data are numbers that contain a decimal. Actually they can also be whole numbers but we’ll gloss over that. •Integers are whole numbers (those numbers without a decimal point). •Logical data take on the value of either TRUE or FALSE. There’s also another special type of logical called NA to represent missing values. 63 64 CHAPTER 3. DATA IN R •Character data are used to represent string values. You can think of character strings as something like a word (or multiple words). A special type of character string is a factor, which is a string but with additional attributes (like levels or an order). We’ll cover factors later. R is (usually) able to automatically distinguish between different classes of data by their nature and the context in which they’re used although you should bear in mind that R can’t actually read your mind and you may have to explicitly tell R how you want to treat a data type. You can find out the type (or class) of any object using the class() function. num <- 2.2 class(num) ## [1] "numeric" char <- "hello" class(char) ## [1] "character" logi <- TRUE class(logi) ## [1] "logical" Alternatively, you can ask if an object is a specific class using using a logical test. The is.[classOfData]() family of functions will return either a TRUE or a FALSE. is.numeric(num) ## [1] TRUE is.character(num) ## [1] FALSE is.character(char) ## [1] TRUE is.logical(logi) ## [1] TRUE It can sometimes be useful to be able to change the class of a variable using the as.[className]() family of coercion functions, although you need to be careful when doing this as you might receive some unexpected results (see what happens below when we try to convert a character string to a numeric). 3.2. DATA STRUCTURES 65 # coerce numeric to character class(num) ## [1] "numeric" num_char <- as.character(num) num_char ## [1] "2.2" class(num_char) ## [1] "character" # coerce character to numeric! class(char) ## [1] "character" char_num <- as.numeric(char) ## Warning: NAs introduced by coercion Here’s a summary table of some of the logical test and coercion functions available to you. Type Logical test Coercing Character is.character as.character Numeric is.numeric as.numeric Logical is.logical as.logical Factor is.factor as.factor Complex is.complex as.complex 3.2 Data structures Now that you’ve been introduced to some of the most important classes of data in R, let’s have a look at some of main structures that we have for storing these data. 3.2.1 Scalars and vectors Perhaps the simplest type of data structure is the vector. You’ve already been introduced to vectors in Chapter 2 although some of the vectors you created only contained a single value. Vectors that have a single value (length 1) are called scalars. Vectors can contain numbers, characters, factors or logicals, but the key thing to remember is that all the elements inside a vector must be of the same class. In other words, vectors can contain either numbers, characters or logicals but not mixtures of these types of data. There is one important exception to this, you can include NA (remember this is special type of logical) to denote missing data in vectors with other data types. 66 CHAPTER 3. DATA IN R 3.2.2 Matrices and arrays Another useful data structure used in many disciplines such as population ecology, theoretical and applied statistics is the matrix. A matrix is simply a vector that has additional attributes called dimensions. Arrays are just multidimensional matrices. Again, matrices and arrays must contain elements all of the same data class. A convenient way to create a matrix or an array is to use the matrix() and array() functions respectively. Below, we will create a matrix from a sequence 1 to 16 in four rows (nrow = 4) and fill the matrix row-wise (byrow = TRUE) rather than the default column-wise. When using the array() function we define the dimensions using the dim =argument, in our case 2 rows, 4 columns in 2 different matrices. my_mat <- matrix(1:16,nrow = 4,byrow = TRUE) my_mat ## [,1] [,2] [,3] [,4] ## [1,] 1 2 3 4 ## [2,] 5 6 7 8 3.2. DATA STRUCTURES 67 ## [3,] 9 10 11 12 ## [4,] 13 14 15 16 my_array <- array(1:16,dim = c(2,4,2)) my_array ##,,1 ## ## [,1] [,2] [,3] [,4] ## [1,] 1 3 5 7 ## [2,] 2 4 6 8 ## ##,,2 ## ## [,1] [,2] [,3] [,4] ## [1,] 9 11 13 15 ## [2,] 10 12 14 16 Sometimes it’s also useful to define row and column names for your matrix but this is not a requirement. To do this use the rownames() and colnames() functions. rownames(my_mat) <- c("A","B","C","D") colnames(my_mat) <- c("a","b","c","d") my_mat ## a b c d ##A1234 ##B5678 ## C 9 10 11 12 ## D 13 14 15 16 Once you’ve created your matrices you can do useful stuff with them and as you’d expect, R has numerous built in functions to perform matrix operations. Some of the most common are given below. For example, to transpose a matrix we use the transposition function t(). my_mat_t <- t(my_mat) my_mat_t ## A B C D ## a 1 5 9 13 ##b261014 ##c371115 ##d481216 To extract the diagonal elements of a matrix and store them as a vector we can use the diag() function. 68 CHAPTER 3. DATA IN R my_mat_diag <- diag(my_mat) my_mat_diag ## [1] 1 6 11 16 The usual matrix addition, multiplication etc can be performed. Note the use of the %*% operator to perform matrix multiplication. mat.1 <- matrix(c(2,0,1,1), nrow = 2)# notice that the matrix has been filled mat.1 # column-wise by default ## [,1] [,2] ## [1,] 2 1 ## [2,] 0 1 mat.2 <- matrix(c(1,1,0,2), nrow = 2) mat.2 ## [,1] [,2] ## [1,] 1 0 ## [2,] 1 2 mat.1 +mat.2 # matrix addition ## [,1] [,2] ## [1,] 3 1 ## [2,] 1 3 mat.1 *mat.2 # element by element products ## [,1] [,2] ## [1,] 2 0 ## [2,] 0 2 mat.1 %*% mat.2 # matrix multiplication ## [,1] [,2] ## [1,] 3 2 ## [2,] 1 2 3.2.3 Lists The next data structure we will quickly take a look at is a list. Whilst vectors and matrices are constrained to contain data of the same type, lists are able to store mixtures of data types. In fact we can even store other data structures such as vectors and arrays within a list or even have a list of a list. This makes for a very flexible data structure which is ideal for storing irregular or non-rectangular data (see Chapter 7 for an example). To create a list we can use the list() function. Note how each of the three list elements are of different classes (character, logical, and numeric) and are of different lengths. 3.2. DATA STRUCTURES 69 list_1 <- list(c("black","yellow","orange"), c(TRUE,TRUE,FALSE,TRUE,FALSE,FALSE), matrix(1:6,nrow = 3)) list_1 ## [[1]] ## [1] "black" "yellow" "orange" ## ## [[2]] ## [1] TRUE TRUE FALSE TRUE FALSE FALSE ## ## [[3]] ## [,1] [,2] ## [1,] 1 4 ## [2,] 2 5 ## [3,] 3 6 Elements of the list can be named during the construction of the list list_2 <- list(colours = c("black","yellow","orange"), evaluation = c(TRUE,TRUE,FALSE,TRUE,FALSE,FALSE), time = matrix(1:6,nrow = 3)) list_2 ## $colours ## [1] "black" "yellow" "orange" ## ## $evaluation ## [1] TRUE TRUE FALSE TRUE FALSE FALSE ## ## $time ## [,1] [,2] ## [1,] 1 4 ## [2,] 2 5 ## [3,] 3 6 or after the list has been created using the names() function. names(list_1) <- c("colours","evaluation","time") list_1 ## $colours ## [1] "black" "yellow" "orange" ## ## $evaluation 70 CHAPTER 3. DATA IN R ## [1] TRUE TRUE FALSE TRUE FALSE FALSE ## ## $time ## [,1] [,2] ## [1,] 1 4 ## [2,] 2 5 ## [3,] 3 6 3.2.4 Data frames Take a look at this video for a quick introduction to data frame objects in R By far the most commonly used data structure to store data in is the data frame. A data frame is a powerful two-dimensional object made up of rows and columns which looks superficially very similar to a matrix. However, whilst matrices are restricted to containing data all of the same type, data frames can contain a mixture of different types of data. Typically, in a data frame each row corresponds to an individual observation and each column corresponds to a different measured or recorded variable. This setup may be familiar to those of you who use LibreOffice Calc or Microsoft Excel to manage and store your data. Perhaps a useful way to think about data frames is that they are essentially made up of a bunch of vectors (columns) with each vector containing its own data type but the data type can be different between vectors. As an example, the data frame below contains the results of an experiment to determine the effect of removing the tip of petunia plants (Petunia sp.) grown at 3 levels of nitrogen on various measures of growth (note: data shown below are a subset of the full dataset). The data frame has 8 variables (columns) and each row represents an individual plant. The variables treat and nitrogen are factors (categorical variables). The treat variable has 2 levels (tip and notip) and the nitrogen level variable has 3 levels (low,medium and high). The variables height,weight,leafarea and shootarea are numeric and the variable flowers is an integer representing the number of flowers. Although the variable block has numeric values, these do not really have any order and could also be treated as a factor (i.e. they could also have been called A and B). There are a couple of important things to bear in mind about data frames. These types of objects are known as rectangular data (or tidy data) as each column must have the same number of observations. Also, any missing data should be recorded as an NA just as we did with our vectors. 3.2. DATA STRUCTURES 71 treat nitrogen block height weight leafarea shootarea flowers tip medium 1 7.5 7.62 11.7 31.9 1 tip medium 1 10.7 12.14 14.1 46.0 10 tip medium 1 11.2 12.76 7.1 66.7 10 tip medium 1 10.4 8.78 11.9 20.3 1 tip medium 1 10.4 13.58 14.5 26.9 4 tip medium 1 9.8 10.08 12.2 72.7 9 notip low 2 3.7 8.10 10.5 60.5 6 notip low 2 3.2 7.45 14.1 38.1 4 notip low 2 3.9 9.19 12.4 52.6 9 notip low 2 3.3 8.92 11.6 55.2 6 notip low 2 5.5 8.44 13.5 77.6 9 notip low 2 4.4 10.60 16.2 63.3 6 We can construct a data frame from existing data objects such as vectors using the data.frame() function. As an example, let’s create three vectors p.height,p.weight and p.names and include all of these vectors in a data frame object called dataf. p.height <- c(180,155,160,167,181) p.weight <- c(65,50,52,58,70) p.names <- c("Joanna","Charlotte","Helen","Karen","Amy") dataf <- data.frame(height = p.height, weight = p.weight, names = p.names) dataf ## height weight names ## 1 180 65 Joanna ## 2 155 50 Charlotte ## 3 160 52 Helen ## 4 167 58 Karen ## 5 181 70 Amy You’ll notice that each of the columns are named with variable name we supplied when we used the data.frame() function. It also looks like the first column of the data frame is a series of numbers from one to five. Actually, this is not really a column but the name of each row. We can check this out by getting R to return the dimensions of the dataf object using the dim() function. We see that there are 5 rows and 3 columns. dim(dataf) # 5 rows and 3 columns ## [1] 5 3 Another really useful function which we use all the time is str() which will return a compact summary of the structure of the data frame object (or any object for that matter). 78 CHAPTER 3. DATA IN R # import .csv file flowers <- read.csv(file = 'data/flower.csv') # import .csv file with dec = "," and sep = ";" flowers <- read.csv2(file = 'data/flower.csv') # import tab delim file with sep = "\t" flowers <- read.delim(file = 'data/flower.txt') You can even import spreadsheet files from MS Excel or other statistics software directly into R but our advice is that this should generally be avoided if possible as it just adds a layer of uncertainty between you and your data. In our opinion it’s almost always better to export your spreadsheets as tab or comma delimited files and then import them into R using the read.table() function. If you’re hell bent on directly importing data from other software you will need to install the foreign package which has functions for importing Minitab, SPSS, Stata and SAS files or the xlsx package to import Excel spreadsheets. 3.3.3 Common import frustrations It’s quite common to get a bunch of really frustrating error messages when you first start importing data into R. Perhaps the most common is Error in file(file, "rt"):cannot open the connection In addition:Warning message: In file(file, "rt"): cannot open file 'flower.txt':No such file or directory This error message is telling you that R cannot find the file you are trying to import. It usually rears its head for one of a couple of reasons (or all of them!). The first is that you’ve made a mistake in the spelling of either the filename or file path. Another common mistake is that you have forgotten to include the file extension in the filename (i.e. .txt). Lastly, the file is not where you say it is or you’ve used an incorrect file path. Using RStudio Projects and having a logical directory structure goes a long way to avoiding these types of errors. Another really common mistake is to forget to include the header = TRUE argument when the first row of the data contains variable names. For example, if we omit this argument when we import our flowers.txt file everything looks OK at first (no error message at least) flowers_bad <- read.table(file = 'data/flower.txt',sep = "\t") 3.3. IMPORTING DATA 79 but when we take a look at our data frame using str() str(flowers_bad) ## 'data.frame': 97 obs. of 8 variables: ## $ V1: chr "treat" "tip" "tip" "tip" ... ## $ V2: chr "nitrogen" "medium" "medium" "medium" ... ## $ V3: chr "block" "1" "1" "1" ... ## $ V4: chr "height" "7.5" "10.7" "11.2" ... ## $ V5: chr "weight" "7.62" "12.14" "12.76" ... ## $ V6: chr "leafarea" "11.7" "14.1" "7.1" ... ## $ V7: chr "shootarea" "31.9" "46" "66.7" ... ## $ V8: chr "flowers" "1" "10" "10" ... We can see an obvious problem, all of our variables have been imported as factors and our variables are named V1,V2,V3 …V8. The problem happens because we haven’t told the read.table() function that the first row contains the variable names and so it treats them as data. As soon as we have a single character string in any of our data vectors, R treats the vectors as character type data (remember all elements in a vector must contain the same type of data). 3.3.4 Other import options There are numerous other functions to import data from a variety of sources and formats. Most of these functions are contained in packages that you will need to install before using them. We list a couple of the more useful packages and functions below. The fread() function from the data.table package is great for importing large data files quickly and efficiently (much faster than the read.table() function). One of the great things about the fread() function is that it will automatically detect many of the arguments you would normally need to specify (like sep = etc). One thing you might need to consider is that the fread() function will return a data.table object by default not a data.frame as would be the case with the read.table() function. This is usually not a problem and you can change this default behaviour by using the argument data.table = FALSE when you use fread() if you prefer a data.frame object. To learn more about the differences between data.table and data.frame objects see here. library(data.table) all_data <- fread(file = 'data/flower.txt') Various functions from the readr package are also very efficient at reading in large data files. The readr package is part of the ‘tidyverse’ collection of packages and provides many equivalent functions to base R for importing data. The readr functions are used in a similar way to the read.table() or read.csv() functions and many of the arguments are the same (see ?readr::read_table for more details). There are however some differences. 80 CHAPTER 3. DATA IN R For example, when using the read_table() function the header = TRUE argument is replaced by col_names = TRUE and the function returns a tibble class object which is the tidyverse equivalent of a data.frame object (see here for differences). library(readr) # import white space delimited files all_data <- read_table(file = 'data/flower.txt',col_names = TRUE) # import comma delimited files all_data <- read_csv(file = 'data/flower.txt') # import tab delimited files all_data <- read_delim(file = 'data/flower.txt',delim = "\t") # or use all_data <- read_tsv(file = 'data/flower.txt') If your data file is ginormous, then the ff and bigmemory packages may be useful as they both contain import functions that are able to store large data in a memory efficient manner. You can find out more about these functions here and here. 3.4 Wrangling data frames Now that you’re able to successfully import your data from an external file into R our next task is to do something useful with our data. Working with data is a fundamental skill which you’ll need to develop and get comfortable with as you’ll likely do a lot of it during any project. The good news is that R is especially good at manipulating, summarising and visualising data. Manipulating data (often known as data wrangling or munging) in R can at first seem a little daunting for the new user but if you follow a few simple logical rules then you’ll quickly get the hang of it, especially with some practice. See this video for a general overview on how to use positional and logical indexes to extract data from a data frame object in R Let’s remind ourselves of the structure of the flowers data frame we imported in the previous section. 3.4. WRANGLING DATA FRAMES 81 flowers <- read.table(file = 'data/flower.txt',header = TRUE,sep = "\t") str(flowers) ## 'data.frame': 96 obs. of 8 variables: ## $ treat : chr "tip" "tip" "tip" "tip" ... ## $ nitrogen : chr "medium" "medium" "medium" "medium" ... ## $ block : int 1 1 1 1 1 1 1 1 2 2 ... ## $ height : num 7.5 10.7 11.2 10.4 10.4 9.8 6.9 9.4 10.4 12.3 ... ## $ weight : num 7.62 12.14 12.76 8.78 13.58 ... ## $ leafarea : num 11.7 14.1 7.1 11.9 14.5 12.2 13.2 14 10.5 16.1 ... ## $ shootarea: num 31.9 46 66.7 20.3 26.9 72.7 43.1 28.5 57.8 36.9 ... ## $ flowers : int 1 10 10 1 4 9 7 6 5 8 ... To access the data in any of the variables (columns) in our data frame we can use the $notation. For example, to access the height variable in our flowers data frame we can use flowers$height. This tells R that the height variable is contained within the data frame flowers. flowers$height ## [1] 7.5 10.7 11.2 10.4 10.4 9.8 6.9 9.4 10.4 12.3 10.4 11.0 7.1 6.0 9.0 ## [16] 4.5 12.6 10.0 10.0 8.5 14.1 10.1 8.5 6.5 11.5 7.7 6.4 8.8 9.2 6.2 ## [31] 6.3 17.2 8.0 8.0 6.4 7.6 9.7 12.3 9.1 8.9 7.4 3.1 7.9 8.8 8.5 ## [46] 5.6 11.5 5.8 5.6 5.3 7.5 4.1 3.5 8.5 4.9 2.5 5.4 3.9 5.8 4.5 ## [61] 8.0 1.8 2.2 3.9 8.5 8.5 6.4 1.2 2.6 10.9 7.2 2.1 4.7 5.0 6.5 ## [76] 2.6 6.0 9.3 4.6 5.2 3.9 2.3 5.2 2.2 4.5 1.8 3.0 3.7 2.4 5.7 ## [91] 3.7 3.2 3.9 3.3 5.5 4.4 This will return a vector of the height data. If we want we can assign this vector to another object and do stuff with it, like calculate a mean or get a summary of the variable using the summary() function. f_height <- flowers$height mean(f_height) ## [1] 6.839583 summary(f_height) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 1.200 4.475 6.450 6.840 9.025 17.200 Or if we don’t want to create an additional object we can use functions ‘on-the-fly’ to only display the value in the console. 82 CHAPTER 3. DATA IN R mean(flowers$height) ## [1] 6.839583 summary(flowers$height) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 1.200 4.475 6.450 6.840 9.025 17.200 Just as we did with vectors, we also can access data in data frames using the square bracket [ ] notation. However, instead of just using a single index, we now need to use two indexes, one to specify the rows and one for the columns. To do this, we can use the notation my_data[rows, columns] where rows and columns are indexes and my_data is the name of the data frame. Again, just like with our vectors our indexes can be positional or the result of a logical test. 3.4.1 Positional indexes To use positional indexes we simple have to write the position of the rows and columns we want to extract inside the [ ]. For example, if for some reason we wanted to extract the first value (1st row ) of the height variable (4th column). flowers[1,4] ## [1] 7.5 # this would give you the same flowers$height[1] ## [1] 7.5 We can also extract values from multiple rows or columns by specifying these indexes as vectors inside the [ ]. To extract the first 10 rows and the first 4 columns we simple supply a vector containing a sequence from 1 to 10 for the rows index (1:10) and a vector from 1 to 4 for the column index (1:4). flowers[1:10,1:4] ## treat nitrogen block height ## 1 tip medium 1 7.5 ## 2 tip medium 1 10.7 ## 3 tip medium 1 11.2 ## 4 tip medium 1 10.4 ## 5 tip medium 1 10.4 ## 6 tip medium 1 9.8 ## 7 tip medium 1 6.9 ## 8 tip medium 1 9.4 ## 9 tip medium 2 10.4 ## 10 tip medium 2 12.3 3.4. WRANGLING DATA FRAMES 83 Or for non sequential rows and columns then we can supply vectors of positions using the c() function. To extract the 1st, 5th, 12th, 30th rows from the 1st, 3rd, 6th and 8th columns. flowers[c(1,5,12,30), c(1,3,6,8)] ## treat block leafarea flowers ## 1 tip 1 11.7 1 ## 5 tip 1 14.5 4 ## 12 tip 2 12.6 6 ## 30 tip 2 11.6 5 All we are doing in the two examples above is creating vectors of positions for the rows and columns that we want to extract. We have done this by using the skills we developed in Chapter 2 when we generated vectors using the c() function or using the :notation. But what if we want to extract either all of the rows or all of the columns? It would be extremely tedious to have to generate vectors for all rows or for all columns. Thankfully R has a shortcut. If you don’t specify either a row or column index in the [ ] then R interprets it to mean you want all rows or all columns. For example, to extract the first 8 rows and all of the columns in the flower data frame flowers[1:8, ] ## treat nitrogen block height weight leafarea shootarea flowers ## 1 tip medium 1 7.5 7.62 11.7 31.9 1 ## 2 tip medium 1 10.7 12.14 14.1 46.0 10 ## 3 tip medium 1 11.2 12.76 7.1 66.7 10 ## 4 tip medium 1 10.4 8.78 11.9 20.3 1 ## 5 tip medium 1 10.4 13.58 14.5 26.9 4 ## 6 tip medium 1 9.8 10.08 12.2 72.7 9 ## 7 tip medium 1 6.9 10.11 13.2 43.1 7 ## 8 tip medium 1 9.4 10.28 14.0 28.5 6 or all of the rows and the first 3 columns. If you’re reading the web version of this book scroll down in output panel to see all of the data. Note, if you’re reading the pdf version of the book some of the output has been truncated to save some space. flowers[, 1:3] ## treat nitrogen block ## 1 tip medium 1 ## 2 tip medium 1 ## 3 tip medium 1 84 CHAPTER 3. DATA IN R ## 4 tip medium 1 ## 5 tip medium 1 ## 6 tip medium 1 ## 7 tip medium 1 ## 8 tip medium 1 ## 9 tip medium 2 ## 10 tip medium 2 ## 87 notip low 1 ## 88 notip low 1 ## 89 notip low 2 ## 90 notip low 2 ## 91 notip low 2 ## 92 notip low 2 ## 93 notip low 2 ## 94 notip low 2 ## 95 notip low 2 ## 96 notip low 2 We can even use negative positional indexes to exclude certain rows and columns. As an example, lets extract all of the rows except the first 85 rows and all columns except the 4th, 7th and 8th columns. Notice we need to use -() when we generate our row positional vectors. If we had just used -1:85 this would actually generate a regular sequence from -1 to 85 which is not what we want (we can of course use -1:-85). flowers[-(1:85), -c(4,7,8)] ## treat nitrogen block weight leafarea ## 86 notip low 1 6.01 17.6 ## 87 notip low 1 9.93 12.0 ## 88 notip low 1 7.03 7.9 ## 89 notip low 2 9.10 14.5 ## 90 notip low 2 9.05 9.6 ## 91 notip low 2 8.10 10.5 ## 92 notip low 2 7.45 14.1 ## 93 notip low 2 9.19 12.4 ## 94 notip low 2 8.92 11.6 ## 95 notip low 2 8.44 13.5 ## 96 notip low 2 10.60 16.2 In addition to using a positional index for extracting particular columns (variables) we can also name the variables directly when using the square bracket [ ] notation. For example, let’s extract the first 5 rows and the variables treat,nitrogen and leafarea. Instead of using flowers[1:5, c(1, 2, 6)] we can instead use 3.4. WRANGLING DATA FRAMES 85 flowers[1:5,c("treat","nitrogen","leafarea")] ## treat nitrogen leafarea ## 1 tip medium 11.7 ## 2 tip medium 14.1 ## 3 tip medium 7.1 ## 4 tip medium 11.9 ## 5 tip medium 14.5 We often use this method in preference to the positional index for selecting columns as it will still give us what we want even if we’ve changed the order of the columns in our data frame for some reason. 3.4.2 Logical indexes Just as we did with vectors, we can also extract data from our data frame based on a logical test. We can use all of the logical operators that we used for our vector examples so if these have slipped your mind maybe pop back and refresh your memory. Let’s extract all rows where height is greater than 12 and extract all columns by default (remember, if you don’t include a column index after the comma it means all columns). big_flowers <- flowers[flowers$height >12, ] big_flowers ## treat nitrogen block height weight leafarea shootarea flowers ## 10 tip medium 2 12.3 13.48 16.1 36.9 8 ## 17 tip high 1 12.6 18.66 18.6 54.0 9 ## 21 tip high 1 14.1 19.12 13.1 113.2 13 ## 32 tip high 2 17.2 19.20 10.9 89.9 14 ## 38 tip low 1 12.3 11.27 13.7 28.7 5 Notice in the code above that we need to use the flowers$height notation for the logical test. If we just named the height variable without the name of the data frame we would receive an error telling us R couldn’t find the variable height. The reason for this is that the height variable only exists inside the flowers data frame so you need to tell R exactly where it is. big_flowers <- flowers[height >12, ] Error in `[.data.frame`(flowers, height >12, ) : object 'height' not found So how does this work? The logical test is flowers$height > 12 and R will only extract those rows that satisfy this logical condition. If we look at the output of just the logical 86 CHAPTER 3. DATA IN R condition you can see this returns a vector containing TRUE if height is greater than 12 and FALSE if height is not greater than 12. flowers$height >12 ## [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE ## [13] FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE ## [25] FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE ## [37] FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE ## [49] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE ## [61] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE ## [73] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE ## [85] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE So our row index is a vector containing either TRUE or FALSE values and only those rows that are TRUE are selected. Other commonly used operators are shown below. flowers[flowers$height >= 6, ] # values greater or equal to 6 flowers[flowers$height <= 6, ] # values less than or equal to 6 flowers[flowers$height == 8, ] # values equal to 8 flowers[flowers$height != 8, ] # values not equal to 8 We can also extract rows based on the value of a character string or factor level. Let’s extract all rows where the nitrogen level is equal to high (again we will output all columns). Notice that the double equals == sign must be used for a logical test and that the character string must be enclosed in either single or double quotes (i.e. "high"). nit_high <- flowers[flowers$nitrogen == "high", ] nit_high ## treat nitrogen block height weight leafarea shootarea flowers ## 17 tip high 1 12.6 18.66 18.6 54.0 9 ## 18 tip high 1 10.0 18.07 16.9 90.5 3 ## 19 tip high 1 10.0 13.29 15.8 142.7 12 ## 20 tip high 1 8.5 14.33 13.2 91.4 5 ## 21 tip high 1 14.1 19.12 13.1 113.2 13 ## 22 tip high 1 10.1 15.49 12.6 77.2 12 ## 23 tip high 1 8.5 17.82 20.5 54.4 3 ## 24 tip high 1 6.5 17.13 24.1 147.4 6 3.4. WRANGLING DATA FRAMES 87 ## 25 tip high 2 11.5 23.89 14.3 101.5 12 ## 26 tip high 2 7.7 14.77 17.2 104.5 4 ## 71 notip high 1 7.2 15.21 15.9 135.0 14 ## 72 notip high 1 2.1 19.15 15.6 176.7 6 ## 73 notip high 2 4.7 13.42 19.8 124.7 5 ## 74 notip high 2 5.0 16.82 17.3 182.5 15 ## 75 notip high 2 6.5 14.00 10.1 126.5 7 ## 76 notip high 2 2.6 18.88 16.4 181.5 14 ## 77 notip high 2 6.0 13.68 16.2 133.7 2 ## 78 notip high 2 9.3 18.75 18.4 181.1 16 ## 79 notip high 2 4.6 14.65 16.7 91.7 11 ## 80 notip high 2 5.2 17.70 19.1 181.1 8 Or we can extract all rows where nitrogen level is not equal to medium (using !=) and only return columns 1 to 4. nit_not_medium <- flowers[flowers$nitrogen != "medium",1:4] nit_not_medium ## treat nitrogen block height ## 17 tip high 1 12.6 ## 18 tip high 1 10.0 ## 19 tip high 1 10.0 ## 20 tip high 1 8.5 ## 21 tip high 1 14.1 ## 22 tip high 1 10.1 ## 23 tip high 1 8.5 ## 24 tip high 1 6.5 ## 25 tip high 2 11.5 ## 26 tip high 2 7.7 ## 87 notip low 1 3.0 ## 88 notip low 1 3.7 ## 89 notip low 2 2.4 ## 90 notip low 2 5.7 ## 91 notip low 2 3.7 ## 92 notip low 2 3.2 ## 93 notip low 2 3.9 ## 94 notip low 2 3.3 ## 95 notip low 2 5.5 ## 96 notip low 2 4.4 We can increase the complexity of our logical tests by combining them with Boolean expressions just as we did for vector objects. For example, to extract all rows where height is greater or equal to 6AND nitrogen is equal to medium AND treat is equal to notip we combine a series of logical expressions with the &symbol. 94 CHAPTER 3. DATA IN R 3.4.4 Adding columns and rows Sometimes it’s useful to be able to add extra rows and columns of data to our data frames. There are multiple ways to achieve this (as there always is in R!) depending on your circumstances. To simply append additional rows to an existing data frame we can use the rbind() function and to append columns the cbind() function. Let’s create a couple of test data frames to see this in action using our old friend the data.frame() function. # rbind for rows df1 <- data.frame(id = 1:4,height = c(120,150,132,122), weight = c(44,56,49,45)) df1 ## id height weight ## 1 1 120 44 ## 2 2 150 56 ## 3 3 132 49 ## 4 4 122 45 df2 <- data.frame(id = 5:6,height = c(119,110), weight = c(39,35)) df2 ## id height weight ## 1 5 119 39 ## 2 6 110 35 df3 <- data.frame(id = 1:4,height = c(120,150,132,122), weight = c(44,56,49,45)) df3 ## id height weight ## 1 1 120 44 ## 2 2 150 56 ## 3 3 132 49 ## 4 4 122 45 df4 <- data.frame(location = c("UK","CZ","CZ","UK")) df4 ## location ## 1 UK ## 2 CZ ## 3 CZ ## 4 UK We can use the rbind() function to append the rows of data in df2 to the rows in df1 and assign the new data frame to df_rcomb. 3.4. WRANGLING DATA FRAMES 95 df_rcomb <- rbind(df1, df2) df_rcomb ## id height weight ## 1 1 120 44 ## 2 2 150 56 ## 3 3 132 49 ## 4 4 122 45 ## 5 5 119 39 ## 6 6 110 35 And cbind to append the column in df4 to the df3 data frame and assign to df_ccomb. df_ccomb <- cbind(df3, df4) df_ccomb ## id height weight location ## 1 1 120 44 UK ## 2 2 150 56 CZ ## 3 3 132 49 CZ ## 4 4 122 45 UK Another situation when adding a new column to a data frame is useful is when you want to perform some kind of transformation on an existing variable. For example, say we wanted to apply a log10 transformation on the height variable in the df_rcomb data frame we created above. We could just create a separate variable to contains these values but it’s good practice to create this variable as a new column inside our existing data frame so we keep all of our data together. Let’s call this new variable height_log10. # log10 transformation df_rcomb$height_log10 <- log10(df_rcomb$height) df_rcomb ## id height weight height_log10 ## 1 1 120 44 2.079181 ## 2 2 150 56 2.176091 ## 3 3 132 49 2.120574 ## 4 4 122 45 2.086360 ## 5 5 119 39 2.075547 ## 6 6 110 35 2.041393 This situation also crops up when we want to convert an existing variable in a data frame from one data class to another data class. For example, the id variable in the df_rcomb data frame is numeric type data (use the str() or class() functions to check for yourself). If we wanted to convert the id variable to a factor to use later in our analysis we can create a new variable called Fid in our data frame and use the factor() function to convert the id variable. 96 CHAPTER 3. DATA IN R # convert to a factor df_rcomb$Fid <- factor(df_rcomb$id) df_rcomb ## id height weight height_log10 Fid ## 1 1 120 44 2.079181 1 ## 2 2 150 56 2.176091 2 ## 3 3 132 49 2.120574 3 ## 4 4 122 45 2.086360 4 ## 5 5 119 39 2.075547 5 ## 6 6 110 35 2.041393 6 str(df_rcomb) ## 'data.frame': 6 obs. of 5 variables: ## $ id : int 1 2 3 4 5 6 ## $ height : num 120 150 132 122 119 110 ## $ weight : num 44 56 49 45 39 35 ## $ height_log10: num 2.08 2.18 2.12 2.09 2.08 ... ## $ Fid : Factor w/ 6 levels "1","2","3","4",..: 1 2 3 4 5 6 3.4.5 Merging data frames Instead of just appending either rows or columns to a data frame we can also merge two data frames together. Let’s say we have one data frame that contains taxonomic information on some common UK rocky shore invertebrates (called taxa) and another data frame that contains information on where they are usually found on the rocky shore (called zone). We can merge these two data frames together to produce a single data frame with both taxonomic and location information. Let’s first create both of these data frames (in reality you would probably just import your different datasets). taxa <- data.frame(GENUS = c("Patella","Littorina","Halichondria","Semibalanus"), species = c("vulgata","littoria","panacea","balanoides"), family = c("patellidae","Littorinidae","Halichondriidae","Archaeobalanidae")) taxa ## GENUS species family ## 1 Patella vulgata patellidae ## 2 Littorina littoria Littorinidae ## 3 Halichondria panacea Halichondriidae ## 4 Semibalanus balanoides Archaeobalanidae zone <- data.frame(genus = c("Laminaria","Halichondria","Xanthoria","Littorina", "Semibalanus","Fucus"), species = c("digitata","panacea","parietina","littoria", "balanoides","serratus"), zone = c("v_low","low","v_high","low_mid","high","low_mid")) 3.4. WRANGLING DATA FRAMES 97 zone ## genus species zone ## 1 Laminaria digitata v_low ## 2 Halichondria panacea low ## 3 Xanthoria parietina v_high ## 4 Littorina littoria low_mid ## 5 Semibalanus balanoides high ## 6 Fucus serratus low_mid Because both of our data frames contains at least one variable in common (species in our case) we can simply use the merge() function to create a new data frame called taxa_zone. taxa_zone <- merge(x = taxa, y = zone) taxa_zone ## species GENUS family genus zone ## 1 balanoides Semibalanus Archaeobalanidae Semibalanus high ## 2 littoria Littorina Littorinidae Littorina low_mid ## 3 panacea Halichondria Halichondriidae Halichondria low Notice that the merged data frame contains only the rows that have species information in both data frames. There are also two columns called GENUS and genus because the merge() function treats these as two different variables that originate from the two data frames. If we want to include all data from both data frames then we will need to use the all = TRUE argument. The missing values will be included as NA. taxa_zone <- merge(x = taxa, y = zone, all = TRUE) taxa_zone ## species GENUS family genus zone ## 1 balanoides Semibalanus Archaeobalanidae Semibalanus high ## 2 digitata <NA> <NA> Laminaria v_low ## 3 littoria Littorina Littorinidae Littorina low_mid ## 4 panacea Halichondria Halichondriidae Halichondria low ## 5 parietina <NA> <NA> Xanthoria v_high ## 6 serratus <NA> <NA> Fucus low_mid ## 7 vulgata Patella patellidae <NA> <NA> If the variable names that you want to base the merge on are different in each data frame (for example GENUS and genus) you can specify the names in the first data frame (known as x) and the second data frame (known as y) using the by.x = and by.y = arguments. 98 CHAPTER 3. DATA IN R taxa_zone <- merge(x = taxa, y = zone, by.x = "GENUS",by.y = "genus",all = TRUE) taxa_zone ## GENUS species.x family species.y zone ## 1 Fucus <NA> <NA> serratus low_mid ## 2 Halichondria panacea Halichondriidae panacea low ## 3 Laminaria <NA> <NA> digitata v_low ## 4 Littorina littoria Littorinidae littoria low_mid ## 5 Patella vulgata patellidae <NA> <NA> ## 6 Semibalanus balanoides Archaeobalanidae balanoides high ## 7 Xanthoria <NA> <NA> parietina v_high Or using multiple variable names. taxa_zone <- merge(x = taxa, y = zone, by.x = c("species","GENUS"), by.y = c("species","genus"), all = TRUE) taxa_zone ## species GENUS family zone ## 1 balanoides Semibalanus Archaeobalanidae high ## 2 digitata Laminaria <NA> v_low ## 3 littoria Littorina Littorinidae low_mid ## 4 panacea Halichondria Halichondriidae low ## 5 parietina Xanthoria <NA> v_high ## 6 serratus Fucus <NA> low_mid ## 7 vulgata Patella patellidae <NA> 3.4.6 Reshaping data frames Reshaping data into different formats is a common task. With rectangular type data (data frames have the same number of rows in each column) there are two main data frame shapes that you will come across: the ‘long’ format (sometimes called stacked) and the ‘wide’ format. An example of a long format data frame is given below. We can see that each row is a single observation from an individual subject and each subject can have multiple rows. This results in a single column of our measurement. long_data <- data.frame( subject = rep(c("A","B","C","D"), each = 3), sex = rep(c("M","F","F","M"), each =3), condition = rep(c("control","cond1","cond2"), times = 4), measurement = c(12.9,14.2,8.7,5.2,12.6,10.1,8.9, 12.1,14.2,10.5,12.9,11.9)) long_data ## subject sex condition measurement 3.4. WRANGLING DATA FRAMES 99 ## 1 A M control 12.9 ## 2 A M cond1 14.2 ## 3 A M cond2 8.7 ## 4 B F control 5.2 ## 5 B F cond1 12.6 ## 6 B F cond2 10.1 ## 7 C F control 8.9 ## 8 C F cond1 12.1 ## 9 C F cond2 14.2 ## 10 D M control 10.5 ## 11 D M cond1 12.9 ## 12 D M cond2 11.9 We can also format the same data in the wide format as shown below. In this format we have multiple observations from each subject in a single row with measurements in different columns (control,cond1 and cond2). This is a common format when you have repeated measurements from sampling units. wide_data <- data.frame(subject = c("A","B","C","D"), sex = c("M","F","F","M"), control = c(12.9,5.2,8.9,10.5), cond1 = c(14.2,12.6,12.1,12.9), cond2 = c(8.7,10.1,14.2,11.9)) wide_data ## subject sex control cond1 cond2 ## 1 A M 12.9 14.2 8.7 ## 2 B F 5.2 12.6 10.1 ## 3 C F 8.9 12.1 14.2 ## 4 D M 10.5 12.9 11.9 Whilst there’s no inherent problem with either of these formats we will sometimes need to convert between the two because some functions will require a specific format for them to work. The most common format is the long format. There are many ways to convert between these two formats but we’ll use the melt() and dcast() functions from the reshape2 package (you will need to install this package first). The melt() function is used to convert from wide to long formats. The first argument for the melt() function is the data frame we want to melt (in our case wide_data). The id.vars = c("subject", "sex") argument is a vector of the variables you want to stack, the measured.vars = c("control", "cond1", "cond2") argument identifies the columns of the measurements in different conditions, the variable.name = "condition" argument specifies what you want to call the stacked column of your different conditions in your output 100 CHAPTER 3. DATA IN R data frame and value.name = "measurement" is the name of the column of your stacked measurements in your output data frame. library(reshape2) wide_data # remind ourselves what the wide format looks like ## subject sex control cond1 cond2 ## 1 A M 12.9 14.2 8.7 ## 2 B F 5.2 12.6 10.1 ## 3 C F 8.9 12.1 14.2 ## 4 D M 10.5 12.9 11.9 # convert wide to long my_long_df <- melt(data = wide_data, id.vars = c("subject","sex"), measured.vars = c("control","cond1","cond2"), variable.name = "condition",value.name = "measurement") my_long_df ## subject sex condition measurement ## 1 A M control 12.9 ## 2 B F control 5.2 ## 3 C F control 8.9 ## 4 D M control 10.5 ## 5 A M cond1 14.2 ## 6 B F cond1 12.6 ## 7 C F cond1 12.1 ## 8 D M cond1 12.9 ## 9 A M cond2 8.7 ## 10 B F cond2 10.1 ## 11 C F cond2 14.2 ## 12 D M cond2 11.9 The dcast() function is used to convert from a long format data frame to a wide format data frame. The first argument is again is the data frame we want to cast (long_data for this example). The second argument is in formula syntax. The subject + sex bit of the formula means that we want to keep these columns separate, and the ~ condition part is the column that contains the labels that we want to split into new columns in our new data frame. The value.var = "measurement" argument is the column that contains the measured data. long_data # remind ourselves what the long format look like ## subject sex condition measurement ## 1 A M control 12.9 ## 2 A M cond1 14.2 ## 3 A M cond2 8.7 ## 4 B F control 5.2 3.5. SUMMARISING DATA FRAMES 101 ## 5 B F cond1 12.6 ## 6 B F cond2 10.1 ## 7 C F control 8.9 ## 8 C F cond1 12.1 ## 9 C F cond2 14.2 ## 10 D M control 10.5 ## 11 D M cond1 12.9 ## 12 D M cond2 11.9 # convert long to wide my_wide_df <- dcast(data = long_data, subject +sex ~condition, value.var = "measurement") my_wide_df ## subject sex cond1 cond2 control ## 1 A M 14.2 8.7 12.9 ## 2 B F 12.6 10.1 5.2 ## 3 C F 12.1 14.2 8.9 ## 4 D M 12.9 11.9 10.5 3.5 Summarising data frames Now that we’re able to manipulate and extract data from our data frames our next task is to start exploring and getting to know our data. In this section we’ll start producing tables of useful summary statistics of the variables in our data frame and in the next two Chapters we’ll cover visualising our data with base R graphics and using the ggplot2 package. A really useful starting point is to produce some simple summary statistics of all of the variables in our flowers data frame using the summary() function. summary(flowers) ## treat nitrogen block height weight ## Length:96 low :32 Min. :1.0 Min. : 1.200 Min. : 5.790 ## Class :character medium:32 1st Qu.:1.0 1st Qu.: 4.475 1st Qu.: 9.027 ## Mode :character high :32 Median :1.5 Median : 6.450 Median :11.395 ## Mean :1.5 Mean : 6.840 Mean :12.155 ## 3rd Qu.:2.0 3rd Qu.: 9.025 3rd Qu.:14.537 ## Max. :2.0 Max. :17.200 Max. :23.890 ## leafarea shootarea flowers ## Min. : 5.80 Min. : 5.80 Min. : 1.000 ## 1st Qu.:11.07 1st Qu.: 39.05 1st Qu.: 4.000 ## Median :13.45 Median : 70.05 Median : 6.000 ## Mean :14.05 Mean : 79.78 Mean : 7.062 102 CHAPTER 3. DATA IN R ## 3rd Qu.:16.45 3rd Qu.:113.28 3rd Qu.: 9.000 ## Max. :49.20 Max. :189.60 Max. :17.000 For numeric variables (i.e. height,weight etc) the mean, minimum, maximum, median, first (lower) quartile and third (upper) quartile are presented. For factor variables (i.e. treat and nitrogen) the number of observations in each of the factor levels is given. If a variable contains missing data then the number of NA values is also reported. If we wanted to summarise a smaller subset of variables in our data frame we can use our indexing skills in combination with the summary() function. For example, to summarise only the height,weight,leafarea and shootarea variables we can include the appropriate column indexes when using the [ ]. Notice we include all rows by not specifying a row index. summary(flowers[, 4:7]) ## height weight leafarea shootarea ## Min. : 1.200 Min. : 5.790 Min. : 5.80 Min. : 5.80 ## 1st Qu.: 4.475 1st Qu.: 9.027 1st Qu.:11.07 1st Qu.: 39.05 ## Median : 6.450 Median :11.395 Median :13.45 Median : 70.05 ## Mean : 6.840 Mean :12.155 Mean :14.05 Mean : 79.78 ## 3rd Qu.: 9.025 3rd Qu.:14.537 3rd Qu.:16.45 3rd Qu.:113.28 ## Max. :17.200 Max. :23.890 Max. :49.20 Max. :189.60 # or equivalently # summary(flowers[, c("height", "weight", "leafarea", "shootarea")]) And to summarise a single variable. summary(flowers$leafarea) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 5.80 11.07 13.45 14.05 16.45 49.20 # or equivalently # summary(flowers[, 6]) As you’ve seen above, the summary() function reports the number of observations in each level of our factor variables. Another useful function for generating tables of counts is the table() function. The table() function can be used to build contingency tables of different combinations of factor levels. For example, to count the number of observations for each level of nitrogen 3.5. SUMMARISING DATA FRAMES 103 table(flowers$nitrogen) ## ## low medium high ## 32 32 32 We can extend this further by producing a table of counts for each combination of nitrogen and treat factor levels. table(flowers$nitrogen, flowers$treat) ## ## notip tip ## low 16 16 ## medium 16 16 ## high 16 16 A more flexible version of the table() function is the xtabs() function. The xtabs() function uses a formula notation (~) to build contingency tables with the cross-classifying variables separated by a +symbol on the right hand side of the formula. xtabs() also has a useful data = argument so you don’t have to include the data frame name when specifying each variable. xtabs(~nitrogen +treat, data = flowers) ## treat ## nitrogen notip tip ## low 16 16 ## medium 16 16 ## high 16 16 We can even build more complicated contingency tables using more variables. Note, in the example below the xtabs() function has quietly coerced our block variable to a factor. xtabs(~nitrogen +treat +block, data = flowers) ## , , block = 1 ## ## treat ## nitrogen notip tip ## low 8 8 ## medium 8 8 ## high 8 8 ## 110 CHAPTER 3. DATA IN R The fwrite() function from the data.table package is very efficient at exporting large data objects and is much faster than the write.table() function. It’s also quite simple to use as it has most of the same arguments as write.table(). To export a tab delimited text file we just need to specify the data frame name, the output file name and file path and the separator between columns. library(data.table) fwrite(flowers_df2, file = 'data/flowers_04_12.txt',sep = "\t") To export a csv delimited file it’s even easier as we don’t even need to include the sep =argument. library(data.table) fwrite(flowers_df2, file = 'data/flowers_04_12.csv') The readr package also comes with two useful functions for quickly writing data to external files: the write_tsv() function for writing tab delimited files and the write_csv() function for saving comma separated values (csv) files. library(readr) write_tsv(flowers_df2, file = 'data/flowers_04_12.txt') write_csv(flowers_df2, file = 'data/flowers_04_12.csv') You can even save data directly to an Excel spreadsheet using the write_excel_csv() function but again we don’t recommend this! 3.7 Exercise 3 Congratulations, you’ve reached the end of Chapter 3! Perhaps now’s a good time to practice some of what you’ve learned. You can find an exercise we’ve prepared for you (and our solutions) on the course website. Chapter 4 Graphics with base R Summarising your data, either numerically or graphically, is an important (if often overlooked) component of any data analysis. Fortunately, R has excellent graphics capabilities and can be used whether you want to produce plots for initial data exploration, model validation or highly complex publication quality figures. There are three main systems for producing graphics in R; base R graphics, lattice graphics and ggplot2. Each of these systems have their strengths and weaknesses and we often use them interchangeably. In this Chapter we’ll focus mostly on base R graphics with a sprinkling of lattice graphics for added variety. In the next Chapter we’ll introduce you to the ggplot2 package. The base R graphics system is the original plotting system that’s been around (and has evolved) since the first days of R. When creating plots with base R we tend to use high level functions (like the plot() function) to first create our plot and then use one or more low level functions (like lines() and text() etc) to add additional information to these plots. This can seem a little weird (and time consuming) when you first start creating fancy plots in R, but it does allow you to customise almost every aspect of your plot and build complexity up in layers. The flip side to this flexibility is that you’ll often need to make many decisions about how you want your plot to look rather than rely on the software to make these decisions for you. Having said that, it’s generally very quick and easy to generate simple exploratory plots with base R graphics. The lattice system is implemented in the lattice() package that comes pre-installed with the standard installation of R. However, it won’t be loaded by default so you’ll first need to use library(lattice) to access all the plotting functions. Unlike base R graphics, lattice plots are mostly generated all in one go using a single function so there’s no need to use high and low level plotting functions to customise the look of a plot. This can be a real advantage as things like margin sizes and plot spacing are adjusted automatically. Lattice plots also make a few more decisions for you about how the plots will look but this comes with a slight cost as customising lattice plots to get them to look exactly how you want can become quite involved. Where lattice plots really shine is plotting complex multi-dimensional data using panel plots (also called trellis plots). We’ll see a couple of examples of these types of plots later in the Chapter. 111 112 CHAPTER 4. GRAPHICS WITH BASE R 4.1 Getting started When you create a plot in RStudio the plot will be displayed in the ‘Plots’ tab by default which is usually located in the bottom right pane in RStudio. You can zoom in on a plot by clicking the ‘Zoom’ button which will display your plot in a separate window. This can be really useful if you have a particularly large or complex plot (we’ve noticed that RStudio sometimes fails to display a plot if it’s ‘big’). You can also scroll through plots you’ve previously created by clicking on one of the ‘arrow’ buttons. 4.2. SIMPLE BASE R PLOTS 113 Plots can also be saved in a variety of formats (pdf, png, tiff, jpeg etc) by clicking on the ‘Export’ button and selecting your desired format. You can also redirect your plots to an external file using R code which we’ll cover later in this Chapter. 4.2 Simple base R plots There are many functions in R to produce plots ranging from the very basic to the highly complex. It’s impossible to cover every aspect of producing graphics in R in this introductory book so we’ll introduce you to most of the common methods of graphing data and describe how to customise your graphs later on in this Chapter. 4.2.1 Scatterplots The most common high level function used to produce plots in R is (rather unsurprisingly) the plot() function. For example, let’s plot the weight of petunia plants from our flowers data frame which we imported in Chapter 3. flowers <- read.table(file = 'data/flower.txt', header = TRUE,sep = "\t", stringsAsFactors = TRUE) plot(flowers$weight) 114 CHAPTER 4. GRAPHICS WITH BASE R 0 20 40 60 80 10 15 20 Index flowers$weight R has plotted the values of weight (on the y axis) against an index since we are only plotting one variable to plot. The index is just the order of the weight values in the data frame (1 first in the data frame and 97 last). The weight variable name has been automatically included as a y axis label and the axes scales have been automatically set. If we’d only included the variable weight rather than flowers$weight, the plot() function will display an error as the variable weight only exists in the flowers data frame object. plot(weight) Error in plot(weight) :object 'weight' not found As many of the base R plotting functions don’t have a data = argument to specify the data frame name directly we can use the with() function in combination with plot() as a shortcut. with(flowers, plot(weight)) To plot a scatterplot of one numeric variable against another numeric variable we just need to include both variables as arguments when using the plot() function. For example to plot shootarea on the y axis and weight of the x axis. plot(x = flowers$weight, y = flowers$shootarea) 4.2. SIMPLE BASE R PLOTS 115 10 15 20 0 50 100 150 flowers$weight flowers$shootarea There is an equivalent approach for these types of plots which often causes some confusion at first. You can also use the formula notation when using the plot() function. However, in contrast to the previous method the formula method requires you to specify the y axis variable first, then a ~and then our x axis variable. plot(flowers$shootarea ~flowers$weight) 10 15 20 0 50 100 150 flowers$weight flowers$shootarea Both of these two approaches are equivalent so we suggest that you just choose the one you prefer and go with it. You can also specify the type of graph you wish to plot using the argument type =. You can plot just the points (type = "p", this is the default), just lines (type = "l"), both points and lines connected (type = "b"), both points and lines with the lines running through the points (type = "o") and empty points joined by lines (type = "c"). For example, let’s use our skills from Chapter 2 to generate two vectors of numbers (my_x and my_y) and then plot one against the other using different type = values to see what type of plots are produced. Don’t worry about the par(mfrow = c(2, 2)) line of code yet. 116 CHAPTER 4. GRAPHICS WITH BASE R We’re just using this to split the plotting device so we can fit all four plots on the same device to save some space. See later in the Chapter for more details about this. The top left plot is type = "l", the top right type = "b", bottom left type = "o" and bottom right is type = "c". my_x <- 1:10 my_y <- seq(from = 1,to = 20,by = 2) par(mfrow = c(2,2)) plot(my_x, my_y, type = "l") plot(my_x, my_y, type = "b") plot(my_x, my_y, type = "o") plot(my_x, my_y, type = "c") 2 4 6 8 10 5 15 my_x my_y 2 4 6 8 10 5 15 my_x my_y 2 4 6 8 10 5 15 my_x my_y 2 4 6 8 10 5 15 my_x my_y Play around: Try to create four more plots by using type = 'p',type = 'h',type = 's', and type = 'n' arguments in plot() function. Admittedly the plots we’ve produced so far don’t look anything particularly special. However, the plot() function is incredibly versatile and can generate a large range of plots which you can customise to your own taste. We’ll cover how to customise plots later in the Chapter. As a quick aside, the plot() function is also what’s known as a generic function which means it can change its default behaviour depending on the type of object used as an argument. You will see an example of this in Chapter 6 where we use the plot() function to generate diagnostic plots of residuals from a linear model object (bet you can’t wait!). 4.2.2 Histograms Frequency histograms are useful when you want to get an idea about the distribution of values in a numeric variable. The hist() function takes a numeric vector as its main argument. Let’s generate a histogram of the height values. 4.2. SIMPLE BASE R PLOTS 117 hist(flowers$height) Histogram of flowers$height flowers$height Frequency 0 5 10 15 0 5 10 15 20 The hist() function automatically creates the breakpoints (or bins) in the histogram using the Sturges formula unless you specify otherwise by using the break = argument. For example, let’s say we want to plot our histogram with breakpoints every 1 cm flower height. We first generate a sequence from zero to the maximum value of height (18 rounded up) in steps of 1 using the seq() function. We can then use this sequence with the breaks = argument. While we’re at it, let’s also replace the ugly title for something a little better using the main = argument brk <- seq(from = 0,to = 18,by = 1) hist(flowers$height, breaks = brk, main = "petunia height") petunia height flowers$height Frequency 0 5 10 15 0 2 4 6 8 10 12 118 CHAPTER 4. GRAPHICS WITH BASE R You can also display the histogram as a proportion rather than a frequency by using the freq = FALSE argument. brk <- seq(from = 0,to = 18,by = 1) hist(flowers$height, breaks = brk, main = "petunia height", freq = FALSE) petunia height flowers$height Density 0 5 10 15 0.00 0.04 0.08 0.12 An alternative to plotting just a straight up histogram is to add a kernel density curve to the plot. You can superimpose a density curve onto the histogram by first using the density() function to compute the kernel density estimates and then use the low level function lines() to add these estimates onto the plot as a line. dens <- density(flowers$height) hist(flowers$height, breaks = brk, main = "petunia height", freq = FALSE) lines(dens) 4.2. SIMPLE BASE R PLOTS 119 petunia height flowers$height Density 0 5 10 15 0.00 0.04 0.08 0.12 4.2.3 Box and violin plots OK, we’ll just come and out and say it, we love boxplots and their close relation the violin plot. Boxplots (or box-and-whisker plots to give them their full name) are very useful when you want to graphically summarise the distribution of a variable, identify potential unusual values and compare distributions between different groups. The reason we love them is their ease of interpretation, transparency and relatively high data-to-ink ratio (i.e. they convey lots of information efficiently). We suggest that you try to use boxplots as much as possible when exploring your data and avoid the temptation to use the more ubiquitous bar plot (even with standard error or 95% confidence intervals bars). The problem with bar plots (aka dynamite plots) is that they hide important information from the reader such as the distribution of the data and assume that the error bars (or confidence intervals) are symmetric around the mean. Of course, it’s up to you what you do but if you’re tempted to use bar plots just Google ‘dynamite plots are evil’ or see here or here for a fuller discussion. To create a boxplot in R we use the boxplot() function. For example, let’s create a boxplot of the variable weight from our flowers data frame. We can also include a y axis label using the ylab = argument. boxplot(flowers$weight, ylab = "weight (g)") 126 CHAPTER 4. GRAPHICS WITH BASE R # or we could use the equivalent # pairs(flowers[, 4:8]) Interpretation of the pairs plot takes a bit of getting used to. The panels on the diagonal give the variable names. The first row of plots displays the height variable on the y axis and the variables weight,leafarea,shootarea and flowers on the x axis for each of the four plots respectively. The next row of plots have weight on the y axis and height, leafarea,shootarea and flowers on the x axis. We interpret the rest of the rows in the same way with the last row displaying the flowers variable on the y axis and the other variables on the x axis. Hopefully you’ll notice that the plots below the diagonal are the same plots as those above the diagonal just with the axis reversed. We can also add additional information to each of our plots by including a panel function when we use the pairs() function. For example, to add a LOWESS (locally weighted scatterplot smoothing) smoother to each of the panels we just need to add the argument panel = panel.smooth. pairs(flowers[, c("height","weight","leafarea", "shootarea","flowers")], panel = panel.smooth) height 10 200 100 5 10 15 10 15 20 weight leafarea 10 30 50 0 50 150 shootarea 5 1510 30 50 5 10 15 5 15 flowers If you take a look at the help file for the pairs() function (?pairs) you’ll find a few more useful panel functions in the ‘Examples’ section. To use these functions you’ll first need to copy the code and paste it into the Console in RStudio (we go into more detail about defining functions in Chapter 7). For example, the panel.cor() function calculates the absolute correlation coefficient between two variables and adjusts the size of the displayed coefficient depending on the value (higher coefficient values are bigger). Don’t worry if you don’t understand this code just yet, for the moment we just need to know how to use it (it might be fun to try and figure it out though!). 4.2. SIMPLE BASE R PLOTS 127 panel.cor <- function(x, y, digits = 2,prefix = "", cex.cor, ...) { usr <- par("usr") par(usr = c(0,1,0,1)) r<- abs(cor(x, y)) txt <- format(c(r, 0.123456789), digits = digits)[1] txt <- paste0(prefix, txt) if(missing(cex.cor)) cex.cor <- 0.8/strwidth(txt) text(0.5,0.5, txt, cex = cex.cor *r) } Once we’ve copied and pasted this code into the Console we can use the panel.cor() function to replace the plots below the diagonal with the correlation coefficient using the lower.panel = argument. pairs(flowers[, c("height","weight","leafarea", "shootarea","flowers")], lower.panel = panel.cor) height 10 20 0.28 0 100 0.14 5 10 15 0.23 10 15 20 weight 0.45 0.66 0.56 leafarea 0.44 10 30 50 0.32 0 50 150 shootarea 0.55 5 1510 30 50 5 10 15 5 15 flowers Another useful function in the ‘Examples’ section is the panel.hist() function. This function generates a histogram of each of the variables in the plot. To use it, we must again copy the code and paste it into the Console in RStudio. panel.hist <- function(x, ...) { usr <- par("usr") 128 CHAPTER 4. GRAPHICS WITH BASE R par(usr = c(usr[1:2], 0,1.5) ) h<- hist(x, plot = FALSE) breaks <- h$breaks; nB <- length(breaks) y<- h$counts; y <- y/max(y) rect(breaks[-nB], 0, breaks[-1], y, col = "cyan", ...) } We can then include it in our pairs() function to place the histograms on the diagonal set of panels using the diag.panel = panel.hist argument. Let’s also apply the panel.smooth() function to the plots above the diagonal while we’re at it using the upper.panel = panel.smooth argument. pairs(flowers[, c("height","weight","leafarea", "shootarea","flowers")], lower.panel = panel.cor, diag.panel = panel.hist, upper.panel = panel.smooth) height 10 20 0.28 0 100 0.14 5 10 15 0.23 10 15 20 weight 0.45 0.66 0.56 leafarea 0.44 10 30 50 0.32 0 50 150 shootarea 0.55 5 1510 30 50 5 10 15 5 15 flowers 4.2.6 Coplots When examining the relationship between two numeric variables, it is often useful to be able to determine whether a third variable is obscuring or changing any relationship. A really handy plot to use in these situations is a conditioning plot (also known as conditional scatterplot plot) which we can create in R by using the coplot() function. The coplot() function plots two variables but each plot is conditioned (|) by a third variable. This third variable can be either numeric or a factor. As an example, let’s look at how the relationship between the number of flowers (flowers variable) and the 4.2. SIMPLE BASE R PLOTS 129 weight of petunia plants changes dependent on leafarea. Note the coplot() function has adata = argument so no need to use the $notation. coplot(flowers ~weight|leafarea, data = flowers) 5 10 15 10 15 20 10 15 20 10 15 20 5 10 15 weight flowers 10 20 30 40 Given : leafarea It takes a little practice to interpret coplots. The number of flowers is plotted on the y axis and the weight of plants on the x axis. The six plots show the relationship between these two variables for different ranges of leaf area. The bar plot at the top indicates the range of leaf area values for each of the plots. The panels are read from bottom left to top right along each row. For example, the bottom left panel shows the relationship between number of flowers and weight for plants with the lowest range of leaf area values (approximately 5 - 11 cm2). The top right plot shows the relationship between flowers and weight for plants with a leaf area ranging from approximately 16 - 50 cm2. Notice that the range of values for leaf area differs between panels and that the ranges overlap from panel to panel. The coplot() function does it’s best to split the data up to ensure there are an adequate number of data points in each panel. If you don’t want to produce plots with overlapping data in the panel you can set the overlap =argument to overlap = 0 coplot(flowers ~weight|leafarea, data = flowers, overlap = 0) 130 CHAPTER 4. GRAPHICS WITH BASE R 5 10 15 10 15 20 10 15 20 10 15 20 5 10 15 weight flowers 10 20 30 40 Given : leafarea You can also use the coplot() function with factor conditioning variables. For example, we can examine the relationship between flowers and weight variables conditioned on the factor nitrogen. The bottom left plot is the relationship between flowers and weight for those plants in the low nitrogen treatment. The top left plot shows the same relationship but for plants in the high nitrogen treatment. coplot(flowers ~weight|nitrogen, data = flowers) 5 10 15 10 15 20 10 15 20 5 10 15 weight flowers low medium high Given : nitrogen We can even use two conditioning variables (either numeric or factors). Let’s look at the relationship between flowers and height but this time condition on both nitrogen and treat. 4.2. SIMPLE BASE R PLOTS 131 coplot(flowers ~weight|nitrogen *treat, data = flowers) 5 10 15 10 15 20 10 15 20 10 15 20 5 10 15 weight flowers low medium high Given : nitrogen notip tip Given : treat The bottom row of plots are for plants in the notip treatment and the top row for plants in the tip treatment. So the bottom left plot shows the relationship between flowers and weight for plants grown in low nitrogen with the notip treatment. The top right plot are those plants grown in high nitrogen with the tip treatment. Similar to the pairs() function we can also apply functions to each of the panels using the panel = argument. For example, let’s add a separate line of best fit from a linear model to each of the panels. Don’t worry about the complicated looking code for now, this example is just included to give you an idea about some of the useful things you can do with base R plotting functions. We will introduce linear models in Chapter 6 and cover how to write functions in Chapter 7. coplot(flowers ~weight|nitrogen *treat, data = flowers, panel = function(x, y, ...) { points(x, y, ...) abline(lm(y ~x), col = "blue")}) 132 CHAPTER 4. GRAPHICS WITH BASE R 5 10 15 10 15 20 10 15 20 10 15 20 5 10 15 weight flowers low medium high Given : nitrogen notip tip Given : treat 4.2.7 Lattice plots Many of the plots we’ve previously created using base R graphics can also be created using functions from the lattice package. For example, we can recreate the frequency histogram of the height variable in our flowers data frame using the histogram() function. All of the plotting functions in the lattice package take the formula notation which is why we need to include ~ height as our first argument. We also need to specify that we want a frequency histogram by using the argument type = "count". Don’t forget to first make the package available using library(lattice). library(lattice) histogram(~height, type = "count",data = flowers) height Count 0 5 10 15 20 0 5 10 15 4.2. SIMPLE BASE R PLOTS 133 Or perhaps we would like a boxplot of our weight variable for each level of nitrogen using the bwplot() function. bwplot(weight ~nitrogen, data = flowers) weight 5 10 15 20 low medium high A (non-exhaustive) list of lattice functions and their base R equivalents is given in the table below. Graph type lattice function Base R function scatterplot xyplot() plot() frequency histogram histogram(type = "count") hist() boxplot bwplot() boxplot() Cleveland dotplot dotplot() dotchart() scatterplot matrix splom() pairs() conditioning plot xyplot(y ~ x | z) coplot() Where lattice plots really come into their own is when we want to plot graphs in multiple panels. For example, let’s plot a histogram of our height variable again but this time create a separate histogram for each nitrogen level. We do this by including the |(pipe) symbol which we read as ‘height conditional on nitrogen level’. Also notice that the axis scales are the same for each of the panels to aid comparison which is the default for lattice plots. histogram(~height |nitrogen, type = "count",data = flowers) 134 CHAPTER 4. GRAPHICS WITH BASE R height Count 0 2 4 6 8 10 0 5 10 15 low 0 5 10 15 medium 0 5 10 15 high If we want to change the layout of our plots we can use the layout = argument. Perhaps we prefer all of the graphs to be stacked one on top of the other in which case we would use layout = c(1, 3) to specify 1 column and 3 rows of plots. histogram(~height |nitrogen, type = "count", layout = c(1,3), data = flowers) height Count 0 2 4 6 8 10 0 5 10 15 low 0 2 4 6 8 10 medium 0 2 4 6 8 10 high We can also easily create conditional boxplots using the same logic. bwplot(weight ~nitrogen |block, data = flowers) 4.2. SIMPLE BASE R PLOTS 135 weight 5 10 15 20 low medium high block low medium high block Notice in the plot above that the block names don’t seem to be displayed (actually they are, the’re just represented as orange vertical bars in the panel name). The reason for this is that our block variable is an integer variable (you can check this with class(flowers$block)) as the blocks were coded as either a 1 or a 2 in the original dataset that we imported into R. We can change this by creating a new variable in our data frame and use the factor() function to convert block to a factor (flowers$Fblock <- factor(flowers$block)) and then use the Fblock variable as the conditioning variable. Or we can just change block to be a factor ‘on-the-fly’ when we use it in the bwplot() function. Note, this doesn’t change the block variable in the flowers data frame, just temporarily when we use the bwplot() function. bwplot(weight ~nitrogen |factor(block), data = flowers) weight 5 10 15 20 low medium high 1 low medium high 2 We can also include more than one conditioning variable. For example, let’s create a scatter plot of height against weight for each level of nitrogen and treat. To do this we’ll use the lattice xyplot() function. 142 CHAPTER 4. GRAPHICS WITH BASE R 0 5 10 15 20 25 30 0 50 100 150 200 weight (g) shoot area (cm2) We can also change the type of plotting symbol, the colour of the symbol and the size of the symbol using the pch =,col = and cex = arguments respectively. The pch = argument takes an integer value between 0 and 25 to define the type of plotting symbol. Symbols 0 to 14 are open symbols, 15 to 20 are filled symbols and 21 to 25 are symbols where you can specify a different fill colour and outside line colour. Here’s a summary table displaying the value and corresponding symbol type. 0123456789 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 The col = argument changes the colour of the plotting symbols. This argument can either take an integer value to specify the colour or a character string giving the colour name. For example, col = "red" changes the plotting symbol to red. To see a list of all 657 preset colours available in base R use the colours() function (you can also use colors()) or perhaps even easier see this link. More colour options are available with other packages (see the excellent RColorBrewer package) or you can even ‘mix’ your own colours using the colorRamp() function (see ?colorRamp for more details). The cex = argument allow you to change the size of the plotting symbol. This argument works in the same way as the other cex arguments we’ ve already seen (i.e. cex.axis) and requires a numeric value to indicate the proportional increase or decrease in size relative to the default value of 1. Let’s change the plotting symbol to a filled circle (16), the colour of the symbol to “dodgerblue1” and decrease the size of the symbol by 10%. 4.3. CUSTOMISING PLOTS 143 par(mar = c(4.1,4.4,4.1,1.9), xaxs = "i",yaxs = "i") plot(flowers$weight, flowers$shootarea, xlab = "weight (g)", ylab = expression(paste("shoot area (cm"^"2",")")), xlim = c(0,30), ylim = c(0,200), bty = "l", las = 1,cex.axis = 0.8,tcl = -0.2, pch = 16,col = "dodgerblue1",cex = 0.9) 0 5 10 15 20 25 30 0 50 100 150 200 weight (g) shoot area (cm2) The last thing we’ll do is add a text label to the plot so we can identify it. Perhaps this plot will be one of a series of plots we want to include in the same figure (see the section on plotting multiple graphs to see how to do this) so it would be nice to be able to refer to it in our figure title. To do this we’ll use the text() function to add a capital ‘A’ to the top right of the plot. The text() function needs an x = and a y = coordinate to position the text, a label = for the text and we can use the cex = argument again to change the size of the text. par(mar = c(4.1,4.4,4.1,1.9), xaxs = "i",yaxs = "i") plot(flowers$weight, flowers$shootarea, xlab = "weight (g)", ylab = expression(paste("shoot area (cm"^"2",")")), xlim = c(0,30), ylim = c(0,200), bty = "l", las = 1,cex.axis = 0.8,tcl = -0.2, pch = 16,col = "dodgerblue1",cex = 0.9) text(x = 28,y = 190,label = "A",cex = 2) 144 CHAPTER 4. GRAPHICS WITH BASE R 0 5 10 15 20 25 30 0 50 100 150 200 weight (g) shoot area (cm2) A We think our plot now looks pretty good so we’ll stop here! There are, however, a multitude of other arguments which you can play around with to change the look of your plots. The best place to quickly look for more information is the help page associated with the par() function (?par) or just do a quick Google search. Here’s a table of the more commonly used arguments. Argument Description adj controls justification of the text (0 left justified, 0.5 centered, 1 right justified) bg specifies the background colour of the plot (i.e. : bg = "red", bg = "blue") bty controls the type of box drawn around the plot, values include: "o","l","7","c","u" ,"]" (the box looks like the corresponding character); if bty = "n" the box is not drawn cex controls the size of text and symbols in the plotting area with respect to the default value of 1. Similar commands include: cex.axis controls the numbers on the axes, cex.lab numbers on the axis labels, cex.main the title and cex.sub the sub-title col controls the colour of symbols; additional argument include: col.axis,col.lab,col.main,col.sub font an integer controlling the style of text (1: normal, 2: bold, 3: italics, 4: bold italics); other argument include font.axis, font.lab,font.main,font.sub las an integer which controls the orientation of the axis labels (0: parallel to the axes, 1: horizontal, 2: perpendicular to the axes, 3: vertical) lty controls the line style, can be an integer (1: solid, 2: dashed, 3: dotted, 4: dotdash, 5: longdash, 6: twodash) 4.3. CUSTOMISING PLOTS 145 Argument Description lwd a numeric which controls the width of lines. Works as per cex pch controls the type of symbol, either an integer between 0 and 25, or any single character within quotes " " ps an integer which controls the size in points of texts and symbols pty a character which specifies the type of the plotting region, “s”: square, “m”: maximal tck a value which specifies the length of tick marks on the axes as a fraction of the width or height of the plot; if tck = 1 a grid is drawn tcl a value which specifies the length of tick marks on the axes as a fraction of the height of a line of text (by default tcl = -0.5) 4.3.2 Building plots For even more control over how your plot looks we can build our plot up in layers, customising each step as we go along. For example, perhaps we want to create a plot of shootarea and weight as we did before but this time we want to change the symbol colours of our data points depending on what level of nitrogen the plants were exposed to. The general approach is to use the high level plotting function plot() to create the general plot (axes, axes labels etc) but without the data points by including the type = "n" argument. We then use the low level function points() to add the plotting symbols for each nitrogen level separately choosing a different colour for each set of points. Let’s go through this approach a step at a time. First we’ll make the plot but suppress plotting the data using the type = "n" argument in the plot() function. par(mar = c(4.1,4.4,4.1,1.9), xaxs = "i",yaxs = "i") plot(flowers$weight, flowers$shootarea, type = "n", xlab = "weight (g)", ylab = expression(paste("shoot area (cm"^"2",")")), xlim = c(0,30), ylim = c(0,200), bty = "l", las = 1,cex.axis = 0.8,tcl = -0.2) 146 CHAPTER 4. GRAPHICS WITH BASE R 0 5 10 15 20 25 30 0 50 100 150 200 weight (g) shoot area (cm2) We can now use the points() function in combination with our square bracket [ ] skills to only select those data from the low level of nitrogen. Whilst using the points() function we can also set the symbol type and the symbol colour using the pch = and col =arguments. par(mar = c(4.1,4.4,4.1,1.9), xaxs = "i",yaxs = "i") plot(flowers$weight, flowers$shootarea, type = "n", xlab = "weight (g)", ylab = expression(paste("shoot area (cm"^"2",")")), xlim = c(0,30), ylim = c(0,200), bty = "l", las = 1,cex.axis = 0.8,tcl = -0.2) points(x = flowers$weight[flowers$nitrogen == "low"], y = flowers$shootarea[flowers$nitrogen == "low"], pch = 16,col = "deepskyblue") 0 5 10 15 20 25 30 0 50 100 150 200 weight (g) shoot area (cm2) 4.3. CUSTOMISING PLOTS 147 We can now use the points() function again to plot data for the medium level of nitrogen and change the symbol colour to something different. Notice that we do not reuse the plot() function here as we are just using the low level function points() to add data points to the existing plot. points(x = flowers$weight[flowers$nitrogen == "medium"], y = flowers$shootarea[flowers$nitrogen == "medium"], pch = 16,col = "yellowgreen") 0 5 10 15 20 25 30 0 50 100 150 200 weight (g) shoot area (cm2) And finally to add the high level of nitrogen data points to the plot and add our text label (‘A’) to the plot as before. points(x = flowers$weight[flowers$nitrogen == "high"], y = flowers$shootarea[flowers$nitrogen == "high"], pch = 16,col = "deeppink3") text(x = 28,y = 190,label = "A",cex = 2) 0 5 10 15 20 25 30 0 50 100 150 200 weight (g) shoot area (cm2) A 148 CHAPTER 4. GRAPHICS WITH BASE R The only thing left to do is to add a legend to the plot to let your reader know what nitrogen level each colour corresponds to. We’ll use another low level function,legend() to do this. The legend() function requires us to provide the x and y coordinates to specify the position of the top left of the legend in the plot, a vector of colours, symbol types and labels to use in the legend. The bty = "n" argument stops a border being drawn around the legend and the title = argument gives the legend a title. leg_cols <- c("deepskyblue","yellowgreen","deeppink3") leg_sym <- c(16,16,16) leg_lab <- c("low","medium","high") legend(x = 1,y = 200,col = leg_cols, pch = leg_sym, legend = leg_lab, bty = "n", title = "Nitrogen level") 0 5 10 15 20 25 30 0 50 100 150 200 weight (g) shoot area (cm2) A Nitrogen level low medium high If you want to see all the code together. par(mar = c(4.1,4.4,4.1,1.9), xaxs="i",yaxs="i") plot(flowers$weight, flowers$shootarea, type = "n", xlab = "weight (g)", ylab = expression(paste("shoot area (cm"^"2",")")), xlim = c(0,30), ylim = c(0,200), bty = "l", las = 1,cex.axis = 0.8,tcl = -0.2) points(x = flowers$weight[flowers$nitrogen == "low"], y = flowers$shootarea[flowers$nitrogen == "low"], pch = 16,col = "deepskyblue") points(x = flowers$weight[flowers$nitrogen == "medium"], y = flowers$shootarea[flowers$nitrogen == "medium"], 4.4. MULTIPLE GRAPHS 149 pch = 16,col = "yellowgreen") points(x = flowers$weight[flowers$nitrogen == "high"], y = flowers$shootarea[flowers$nitrogen == "high"], pch = 16,col = "deeppink3") text(x = 28,y = 190,label = "A",cex = 2) leg_cols <- c("deepskyblue","yellowgreen","deeppink3") leg_sym <- c(16,16,16) leg_lab <- c("low","medium","high") legend(x = 1,y = 200,col = leg_cols, pch = leg_sym, legend = leg_lab, bty = "n", title = "Nitrogen level") The table below highlights some of the low level potting functions you might find useful. Function Description lines() add connected lines to a plot curve() draws a curve corresponding to a function arrows() draws arrows between 2 points text() adds text to a plot mtext() adds text to one of the 4 plot margins axis() adds an axis to the current plot rect() draws a rectangle legend() adds a legend to the plot points() adds points to the plot abline() adds a straight line to a plot grid() adds a rectangular grid to the current plot polygon() draws a polygon 4.4 Multiple graphs There are a number of different methods for plotting multiple graphs within the same graphics device, some of which you’ve already met such as pairs(),coplot(),xyplot() etc. However these functions rely on plotting multiple graphs in different panels within the same plot. If you want to plot separate plots within the same graphics device you’ll need a different approach. One of the most common methods is to use the main graphical function par() to split the plotting device up into a number of defined sections using the mfrow = argument. With this method, you first need to specify the number of rows and columns of plots you would like and then run the code for each plot. For 150 CHAPTER 4. GRAPHICS WITH BASE R example, to plot two graphs side by side we would use par(mfrow = c(1, 2)) to split the device into 1 row and two columns. par(mfrow = c(1,2)) plot(flowers$weight, flowers$shootarea, xlab = "weight", ylab = "shoot area") boxplot(shootarea ~nitrogen, data = flowers, cex.axis = 0.6) 10 15 20 0 50 100 150 weight shoot area low medium high 0 50 100 150 nitrogen shootarea Or if we wanted to plot four plots we can split our plotting device into 2 rows and 2 columns. par(mfrow = c(2,2)) plot(flowers$weight, flowers$shootarea, xlab = "weight", ylab = "shoot area") boxplot(shootarea ~nitrogen, cex.axis = 0.8,data = flowers) hist(flowers$weight, main ="") dotchart(flowers$weight) 10 15 20 0 100 weight shoot area low medium high 0 100 nitrogen shootarea flowers$weight Frequency 5 10 15 20 0 10 20 10 15 20 4.4. MULTIPLE GRAPHS 151 Once you’ve finished making your plots don’t forget to reset your plotting device back to normal with par(mfrow = c(1, 1)). A more flexible approach is to use the layout() function. The layout() function allows you to split your plotting device up into different sized regions and can be used to build complex figures. Before using the layout() function we first need to specify how we’re going to split our plotting device by creating a matrix using the matrix() function (see Chapter 2 to remind yourself). Let’s create a 2 x 2 matrix. layout_mat <- matrix(c(2,0,1,3), nrow = 2,ncol = 2, byrow = TRUE) layout_mat ## [,1] [,2] ## [1,] 2 0 ## [2,] 1 3 The matrix above represents splitting the plotting device into 2 rows and 2 columns. The first plot will occupy the lower left panel, the second plot the upper left panels and the third plot the lower right panel. The upper right panel will not contain a plot as we have placed a zero here. We can now use the layout() function to define our layout. As we have two rows and two columns we need to specify the height of each row and the width of each column using the heights = and widths = arguments. The respect = TRUE argument ensures that the units used to define the widths are the same as those to define the heights. We can get a graphical representation of our layout by using the layout.show() function. my_lay <- layout(mat = layout_mat, heights = c(1,3), widths = c(3,1), respect =TRUE) layout.show(my_lay) 254 CHAPTER 6. SIMPLE STATISTICS IN R text. Further information can be found in any of the excellent documents referred to in Chapter 2. A table of some of the more common statistical functions is given below to get you started. R function Use glm() Fit a generalised linear model with a specific error structure specified using the family = argument (Poisson, binomial, gamma) gam() Fit a generalised additive model. The R package mgcv must be loaded lme() &nlme() Fit linear and non-linear mixed effects models. The R package nlme must be loaded lmer() Fit linear and generalised linear and non-linear mixed effects models. The package lme4 must be installed and loaded gls() Fit generalised least squares models. The R package nlme must be loaded kruskal.test() Performs a Kruskal-Wallis rank sum test friedman.test() Performs a Friedman’s test 6.5 Exercise 6 Congratulations, you’ve reached the end of Chapter 6! Perhaps now’s a good time to practice some of what you’ve learned. You can find an exercise we’ve prepared for you (and our solutions) on the course website. Chapter 7 Programming in R After learning the basics, programming in R is the next big step towards attaining coding nirvana. In R this freedom is gained by being able to wield and manipulate code to get it to do exactly what you want, rather than contort, bend and restrict yourself to what others have built. It’s worth remembering that many packages are written with the goal of solving a problem the author had. Your problem may be similar, but not exactly the same. In such cases you could contort, bend and restrict your data so that it works with the package, or you could build your own functions that solve your specific problems. But there are already a vast number of R packages available, surely more than enough to cover everything you could possibly want to do? Why then, would you ever need to use R as a programming language? Why not just stick to the functions from a package? Well, in some cases you’ll want to customise those existing functions to suit your specific needs. Or you may want to implement an approach that’s hot off the press or you’ve come up with an entirely novel idea, which means there won’t be any preexisting packages that work for you. Both of these are not particularly common early-on when you start using R, but you may well find as you progress, the reliance on existing functions becomes a little limiting. In addition, if you ever start using more advanced statistical approaches (i.e. Bayesian inference coded through Stan or JAGS) then an understanding of basic programming constructs, especially loops, becomes fundamental. In this Chapter we’ll explore the basics of the programming side of R. Even if you never create your own function, you will at the very least use functions on a daily basis. Pulling back the curtain and showing the nitty-gritty of how these work will hopefully, in the worst case, help your confidence and in the best case, provide a starting point for doing some clever coding. 7.1 Looking behind the curtain A good way to start learning to program in R is to see what others have done. We can start by briefly peeking behind the curtain. In Chapter 5 we made use of the theme_classic() function when customising our ggplot figures. With many functions in R, if you want to have a quick glance at the machinery behind the scenes, we can simply 255 256 CHAPTER 7. PROGRAMMING IN R write the function name but without the (). This is the same trick we used in Chapter 5to alter the theme_classic() style of ggplot2 to make theme_rbook(). Note that to view the source code of base R packages (those that come with R) requires some additional steps which we won’t cover here (see this link if you’re interested), but for most other packages that you install yourself, generally entering the function name without () will show the source code of the function. theme_classic ## function (base_size = 11, base_family = "", base_line_size = base_size/22, ## base_rect_size = base_size/22) ## { ## theme_bw(base_size = base_size, base_family = base_family, ## base_line_size = base_line_size, base_rect_size = base_rect_size) %+replace% ## theme(panel.border = element_blank(), panel.grid.major = element_blank(), ## panel.grid.minor = element_blank(), axis.line = element_line(colour = "black", ## linewidth = rel(1)), strip.background = element_rect(fill = "white", ## colour = "black", linewidth = rel(2)), complete = TRUE) ## } ## <bytecode: 0x7f8d36b0fbd8> ## <environment: namespace:ggplot2> What we see above is the underlying code for this particular function. As we did in Chapter 5, we could copy and paste this into our own script and make any changes we deemed necessary. This approach isn’t limited to ggplot2 functions and can be used for other functions as well, although tread carefully and test the changes you’ve made. Don’t worry overly if most of the code contained in functions doesn’t make sense immediately. This will be especially true if you are new to R, in which case it seems incredibly intimidating. To help with that, we’ll begin by making our own functions in R in the next section. 7.2 Functions in R Functions are your loyal servants, waiting patiently to do your bidding to the best of their ability. They’re made with the utmost care and attention … though sometimes may end up being something of a Frankenstein’s monster - with an extra limb or two and a head put on backwards. But no matter how ugly they may be they’re completely faithful to you. They’re also very stupid. If we asked you to go to the supermarket to get us some ingredients to make Francesinha, even if you don’t know what the heck that is, you’d be able to guess and bring at least something back. Or you could decide to make something else. Or you could ask a celebrity chef for help. Or you could pull out your phone and search online for what 7.2. FUNCTIONS IN R 257 Francesinha is. The point is, even if we didn’t give you enough information to do the task, you’re intelligent enough to, at the very least, try to find a work around. If instead, we asked our loyal function to do the same, it would listen intently to our request, stand still for a few milliseconds, compose itself, and then start shouting Error: 'data' must be a data frame, or other object .... It would then repeat this every single time we asked it to do the job. The point here, is that code and functions are not intelligent. They cannot find workarounds. It’s totally reliant on you, to tell it very explicitly what it needs to do step by step. Remember two things: the intelligence of code comes from the coder, not the computer and functions need exact instructions to work. To prevent functions from being too stupid you must provide the information the function needs in order for it to function. As with the Francesinha example, if we’d supplied a recipe list to the function, it would have managed just fine. We call this “fulfilling an argument”. The vast majority of functions require the user to fulfill at least one argument. This can be illustrated in the pseudocode below. When we make a function we can specify what arguments the user must fulfill (e.g. argument1 and argument2), as well as what to do once it has this information (expression): nameOfFunction <- function(argument1, argument2, ...) {expression} The first thing to note is that we’ve used the function function() to create a new function called nameOfFunction. To walk through the above code; we’re creating a function called nameOfFunction. Within the round brackets we specify what information (i.e. arguments) the function requires to run (as many or as few as needed). These arguments are then passed to the expression part of the function. The expression can be any valid R command or set of R commands and is usually contained between a pair of braces { } (if a function is only one line long you can omit the braces). Once you run the above code, you can then use your new function by typing: nameOfFunction(argument1, argument2) Confused? Let’s work through an example to help clear things up. First we are going to create a data frame called city, where columns porto,aberdeen, nairobi, and genoa are filled with 100 random values drawn from a bag (using the rnorm() function to draw random values from a Normal distribution with mean 0 and standard deviation of 1). We also include a “problem”, for us to solve later, by including 10 NA values within the nairobi column (using rep(NA, 10)). 258 CHAPTER 7. PROGRAMMING IN R city <- data.frame( porto = rnorm(100), aberdeen = rnorm(100), nairobi = c(rep(NA,10), rnorm(90)), genoa = rnorm(100) ) Let’s say that you want to multiply the values in the variables Porto and Aberdeen and create a new object called porto_aberdeen. We can do this “by hand” using: porto_aberdeen <- city$porto *city$aberdeen We’ve now created an object called porto_aberdeen by multiplying the vectors city$porto and city$aberdeen. Simple. If this was all we needed to do, we can stop here. R works with vectors, so doing these kinds of operations in R is actually much simpler than other programming languages, where this type of code might require loops (we say that R is a vectorised language). Something to keep in mind for later is that doing these kinds of operations with loops can be much slower compared to vectorisation. But what if we want to repeat this multiplication many times? Let’s say we wanted to multiply columns porto and aberdeen,aberdeen and genoa, and nairobi and genoa. In this case we could copy and paste the code, replacing the relevant information. porto_aberdeen <- city$porto *city$aberdeen aberdeen_genoa <- city$aberdeen *city$aberdeen nairobi_genoa <- city$nairobi *city$genoa While this approach works, it’s easy to make mistakes. In fact, here we’ve “forgotten” to change aberdeen to genoa in the second line of code when copying and pasting. This is where writing a function comes in handy. If we were to write this as a function, there is only one source of potential error (within the function itself) instead of many copy-pasted lines of code (which we also cut down on by using a function). In this case, we’re using some fairly trivial code where it’s maybe hard to make a genuine mistake. But what if we increased the complexity? city$porto *city$aberdeen /city$porto +(city$porto *10^(city$aberdeen)) -city$aberdeen -(city$porto *sqrt(city$aberdeen +10)) Now imagine having to copy and paste this three times, and in each case having to change the porto and aberdeen variables (especially if we had to do it more than three times). 7.2. FUNCTIONS IN R 259 What we could do instead is generalise our code for xand ycolumns instead of naming specific cities. If we did this, we could recycle the x * y code. Whenever we wanted to multiple columns together, we assign a city to either xor y. We’ll assign the multiplication to the objects porto_aberdeen and aberdeen_nairobi so we can come back to them later. # Assign x and y values x<- city$porto y<- city$aberdeen # Use multiplication code porto_aberdeen <- x*y # Assign new x and y values x<- city$aberdeen y<- city$nairobi # Reuse multiplication code aberdeen_nairobi <- x*y This is essentially what a function does. OK down to business, let’s call our new function multiply_columns() and define it with two arguments, xand y. In the function code we simply return the value of x * y using the return() function. Using the return() function is not strictly necessary in this example as R will automatically return the value of the last line of code in our function. We include it here to make this explicit. multiply_columns <- function(x, y) { return(x *y) } Now that we’ve defined our function we can use it. Let’s use the function to multiple the columns city$porto and city$aberdeen and assign the result to a new object called porto_aberdeen_func. porto_aberdeen_func <- multiply_columns(x = city$porto, y = city$aberdeen) porto_aberdeen_func ## [1] -0.0506197327 -0.4432141350 0.3108201105 -0.2919239091 -1.1366253825 ## [6] -0.3399143242 0.0386067903 -0.0544747104 0.0006411174 -2.1307848846 ## [11] -1.2609183635 0.1331581979 -3.3295965306 0.5764768687 0.0781312978 ## [16] -0.0437793593 -0.3253823630 -0.3572483727 -0.2340859154 0.0339204099 ## [21] 0.5439459974 0.6183965126 -0.2428913302 0.0403144399 0.0435016842 ## [26] 0.7049567566 -0.1453578744 0.3484937168 -3.7316204837 -0.0088017265 260 CHAPTER 7. PROGRAMMING IN R ## [31] -0.2376784823 -0.0902000868 0.1179349916 -0.0215942052 -0.3865748654 ## [36] -0.3914259779 -0.3329847718 -0.0355957735 4.0772656249 0.2729821809 ## [41] -0.1704451143 0.1002800649 0.6687341403 -0.4311306371 -2.0069960118 ## [46] -1.0100703945 0.1429819387 -0.2577061965 0.0234090642 -0.2903305085 ## [51] 1.5468207129 0.5075457731 -0.6612056716 0.7181899311 -0.0240039699 ## [56] 0.1513975807 -0.0747482740 -1.4085669960 -0.5782803178 0.4623300714 ## [61] 0.5931229059 1.6677806429 -0.6167697646 -0.0605397617 -0.6233045972 ## [66] -0.0808670626 1.1602354255 -1.1427446696 2.5554826788 0.1857713171 ## [71] -1.5619418626 0.2050820137 0.2140668501 0.6101150976 0.5710091209 ## [76] 0.3790092719 -0.1167624827 -0.8736133682 0.0613002222 0.3692599284 ## [81] -0.0178162316 -0.8446248106 0.1913479908 -0.4570263770 3.3978891988 ## [86] 0.2635802481 0.2436495629 -0.6944883151 1.0399681629 0.7000514854 ## [91] 1.5209518248 -0.6275459443 1.3351369461 0.1898882544 1.1237113915 ## [96] -0.3917062564 1.1856465899 -0.1463636917 0.0132100567 0.0496237979 If we’re only interested in multiplying city$porto and city$aberdeen, it would be overkill to create a function to do something once. However, the benefit of creating a function is that we now have that function added to our environment which we can use as often as we like. We also have the code to create the function, meaning we can use it in completely new projects, reducing the amount of code that has to be written (and retested) from scratch each time. As a rule of thumb, you should consider writing a function whenever you’ve copied and pasted a block of code more than twice. To satisfy ourselves that the function has worked properly, we can compare the porto_aberdeen variable with our new variable porto_aberdeen_func using the identical() function. The identical() function tests whether two objects are exactly identical and returns either a TRUE or FALSE value. Use ?identical if you want to know more about this function. identical(porto_aberdeen, porto_aberdeen_func) ## [1] TRUE And we confirm that the function has produced the same result as when we do the calculation manually. We recommend getting into a habit of checking that the function you’ve created works the way you think it has. Now let’s use our multiply_columns() function to multiply columns aberdeen and nairobi. Notice now that argument xis given the value city$aberdeen and ythe value city$nairobi. aberdeen_nairobi_func <- multiply_columns(x = city$aberdeen, y = city$nairobi) aberdeen_nairobi_func ## [1] NA NA NA NA NA ## [6] NA NA NA NA NA ## [11] -1.818835640 0.069272301 0.018907325 -0.348491292 0.713220342 7.2. FUNCTIONS IN R 261 ## [16] 0.617257230 -4.594103324 1.688394991 -0.279758221 0.008449800 ## [21] -0.336269721 0.284820505 0.001846761 0.015733845 -0.378288644 ## [26] 0.829339248 -1.504604685 0.404052839 -0.233627807 -0.159035890 ## [31] -1.638663797 0.947715298 -0.960145855 -0.020679843 1.362055806 ## [36] -0.648757823 -0.023641242 0.180527916 -0.017370698 -0.101936670 ## [41] -0.159893010 -0.638490853 1.765985608 0.010745759 -1.240070109 ## [46] -1.700468333 -0.183591396 -0.601803446 -0.216330706 -0.371921944 ## [51] -0.234472164 -0.043227622 -1.536227571 -0.583180162 -0.182803701 ## [56] -0.579301422 -0.100975934 -1.570994085 -0.204840043 2.201432248 ## [61] 1.015200239 0.832711440 0.203847374 -0.331274664 -1.088656144 ## [66] -0.392745474 2.224547956 1.189232601 1.108834420 0.345535354 ## [71] -0.664589864 1.046810343 0.046499349 -0.989949412 -0.107914610 ## [76] 0.376635421 -0.087812899 -0.826459387 -0.057487138 0.121719362 ## [81] -0.137320600 0.328071801 0.022542978 -0.389179273 0.191388409 ## [86] 0.119479952 -0.847346250 -0.955426387 0.497962007 2.322256787 ## [91] 1.539839313 -0.623016575 -1.777796553 0.965421151 3.169734617 ## [96] 0.096019019 -0.320122350 -0.003816275 0.074307573 -0.142337877 So far so good. All we’ve really done is wrapped the code x * y into a function, where we ask the user to specify what their xand yvariables are. Now let’s add a little complexity. If you look at the output of nairobi_genoa some of the calculations have produced NA values. This is because of those NA values we included in nairobi when we created the city data frame. Despite these NA values, the function appeared to have worked but it gave us no indication that there might be a problem. In such cases we may prefer if it had warned us that something was wrong. How can we get the function to let us know when NA values are produced? Here’s one way. multiply_columns <- function(x, y) { temp_var <- x*y if (any(is.na(temp_var))) { warning("The function has produced NAs") return(temp_var) }else { return(temp_var) } } aberdeen_nairobi_func <- multiply_columns(city$aberdeen, city$nairobi) ## Warning in multiply_columns(city$aberdeen, city$nairobi): The function has ## produced NAs porto_aberdeen_func <- multiply_columns(city$porto, city$aberdeen) The core of our function is still the same. We still have x * y, but we’ve now got an extra six lines of code. Namely, we’ve included some conditional statements, if and 262 CHAPTER 7. PROGRAMMING IN R else, to test whether any NAs have been produced and if they have we display a warning message to the user. The next section of this Chapter will explain how these work and how to use them. 7.3 Conditional statements x*ydoes not apply any logic. It merely takes the value of xand multiplies it by the value of y. Conditional statements are how you inject some logic into your code. The most commonly used conditional statement is if. Whenever you see an if statement, read it as ‘If X is TRUE, do a thing’. Including an else statement simply extends the logic to ‘If X is TRUE, do a thing, or else do something different’. Both the if and else statements allow you to run sections of code, depending on a condition is either TRUE or FALSE. The pseudocode below shows you the general form. if (condition) { Code executed when condition is TRUE }else { Code executed when condition is FALSE } To delve into this a bit more, we can use an old programmer joke to set up a problem. A programmer’s partner says: ‘Please go to the store and buy a carton of milk and if they have eggs, get six.’ The programmer returned with 6 cartons of milk. When the partner sees this, and exclaims ‘Why the heck did you buy six cartons of milk?’ The programmer replied ‘They had eggs’. At the risk of explaining a joke, the conditional statement here is whether or not the store had eggs. If coded as per the original request, the programmer should bring 6 cartons of milk if the store had eggs (condition = TRUE), or else bring 1 carton of milk if there weren’t any eggs (condition = FALSE). In R this is coded as: 7.3. CONDITIONAL STATEMENTS 263 eggs <- TRUE # Whether there were eggs in the store if (eggs == TRUE) { # If there are eggs n.milk <- 6# Get 6 cartons of milk }else {# If there are not eggs n.milk <- 1# Get 1 carton of milk } We can then check n.milk to see how many milk cartons they returned with. n.milk ## [1] 6 And just like the joke, our R code has missed that the condition was to determine whether or not to buy eggs, not more milk (this is actually a loose example of the Winograd Scheme, designed to test the intelligence of artificial intelligence by whether it can reason what the intended referent of a sentence is). We could code the exact same egg-milk joke conditional statement using an ifelse() function. eggs <- TRUE n.milk <- ifelse(eggs == TRUE,yes = 6,no = 1) This ifelse() function is doing exactly the same as the more fleshed out version from earlier, but is now condensed down into a single line of code. It has the added benefit of working on vectors as opposed to single values (more on this later when we introduce loops). The logic is read in the same way; “If there are eggs, assign a value of 6 to n.milk, if there isn’t any eggs, assign the value 1 to n.milk”. We can check again to make sure the logic is still returning 6 cartons of milk: n.milk ## [1] 6 Currently we’d have to copy and paste code if we wanted to change if eggs were in the store or not. We learned above how to avoid lots of copy and pasting by creating a function. Just as with the simple x*yexpression in our previous multiply_columns() function, the logical statements above are straightforward to code and well suited to be turned into a function. How about we do just that and wrap this logical statement up in a function? 270 CHAPTER 7. PROGRAMMING IN R Again, it’s a good idea to test that we are getting something sensible from our loop (remember, check, check and check again!). To do this we can use the identical() function to compare the variables we created by hand with each iteration of the loop manually. porto_aberdeen_func <- multiply_columns(city$porto, city$aberdeen) i<- 1 identical(multiply_columns(city[, i], city[, i +1]), porto_aberdeen_func) ## [1] TRUE aberdeen_nairobi_func <- multiply_columns(city$aberdeen, city$nairobi) ## Warning in multiply_columns(city$aberdeen, city$nairobi): The function has ## produced NAs i<- 2 identical(multiply_columns(city[, i], city[, i +1]), aberdeen_nairobi_func) ## Warning in multiply_columns(city[, i], city[, i + 1]): The function has ## produced NAs ## [1] TRUE If you can follow the examples above, you’ll be in a good spot to begin writing some of your own for loops. That said there are other types of loops available to you. 7.5.2 While loop Another type of loop that you may use (albeit less frequently) is the while loop. The while loop is used when you want to keep looping until a specific logical condition is satisfied (contrast this with the for loop which will always iterate through an entire sequence). The basic structure of the while loop is: while(logical_condition){ expression } A simple example of a while loop is: i<- 0 while (i <= 4) { i<- i+1 print(i) } ## [1] 1 ## [1] 2 7.5. LOOPS 271 ## [1] 3 ## [1] 4 ## [1] 5 Here the loop will only continue to pass values to the main body of the loop (the expression body) when iis less than or equal to 4 (specified using the <= operator in this example). Once iis greater than 4 the loop will stop. There is another, very rarely used type of loop; the repeat loop. The repeat loop has no conditional check so can keep iterating indefinitely (meaning a break, or “stop here”, has to be coded into it). It’s worthwhile being aware of it’s existence, but for now we don’t think you need to worry about it; the for and while loops will see you through the vast majority of your looping needs. 7.5.3 When to use a loop? Loops are fairly commonly used, though sometimes a little overused in our opinion. Equivalent tasks can be performed with functions, which are often more efficient than loops. Though this raises the question when should you use a loop? In general loops are implemented inefficiently in R and should be avoided when better alternatives exist, especially when you’re working with large datasets. However, loop are sometimes the only way to achieve the result we want. Some examples of when using loops can be appropriate: • Some simulations (e.g. the Ricker model can, in part, be built using loops) • Recursive relationships (a relationship which depends on the value of the previous relationship [“to understand recursion, you must understand recursion”]) • More complex problems (e.g., how long since the last badger was seen at site 𝑗, given a pine marten was seen at time 𝑡, at the same location 𝑗as the badger, where the pine marten was detected in a specific 6 hour period, but exclude badgers seen 30 minutes before the pine marten arrival, repeated for all pine marten detections) • While loops (keep jumping until you’ve reached the moon) 7.5.4 If not loops, then what? In short, use the apply family of functions; apply(),lapply(),tapply(),sapply(),vapply(), and mapply(). The apply functions can often do the tasks of most “home-brewed” loops, sometimes faster (though that won’t really be an issue for most people) but more importantly with a much lower risk of error. A strategy to have in the back of your mind which may be useful is; for every loop you make, try to remake it using an apply function (often lapply or sapply will work). If you can, use the apply version. There’s 272 CHAPTER 7. PROGRAMMING IN R nothing worse than realising there was a small, tiny, seemingly meaningless mistake in a loop which weeks, months or years down the line has propagated into a huge mess. We strongly recommend trying to use the apply functions whenever possible. lapply Your go to apply function will often be lapply() at least in the beginning. The way that lapply() works, and the reason it is often a good alternative to for loops, is that it will go through each element in a list and perform a task (i.e. run a function). It has the added benefit that it will output the results as a list - something you’d have to otherwise code yourself into a loop. An lapply() has the following structure: lapply(X, FUN) Here Xis the vector which we want to do something to. FUN stands for how much fun this is (just kidding!). It’s also short for “function”. Let’s start with a simple demonstration first. Let’s use the lapply() function create a sequence from 1 to 5 and add 1 to each observation (just like we did when we used a for loop): lapply(0:4,function(a) {a +1}) ## [[1]] ## [1] 1 ## ## [[2]] ## [1] 2 ## ## [[3]] ## [1] 3 ## ## [[4]] ## [1] 4 ## ## [[5]] ## [1] 5 Notice that we need to specify our sequence as 0:4 to get the output 1 ,2 ,3 ,4 , 5 as we are adding 1to each element of the sequence. See what happens if you use 1:5 instead. Equivalently, we could have defined the function first and then used the function in lapply(). 7.6. EXERCISE 7 273 add_fun <- function(a) {a +1} lapply(0:4, add_fun) ## [[1]] ## [1] 1 ## ## [[2]] ## [1] 2 ## ## [[3]] ## [1] 3 ## ## [[4]] ## [1] 4 ## ## [[5]] ## [1] 5 The sapply() function does the same thing as lapply() but instead of storing the results as a list, it stores them as a vector. sapply(0:4,function(a) {a +1}) ##[1]12345 As you can see, in both cases, we get exactly the same results as when we used the for loop. 7.6 Exercise 7 Congratulations, you’ve reached the end of Chapter 7! Perhaps now’s a good time to practice some of what you’ve learned. You can find an exercise we’ve prepared for you (and our solutions) on the course website. 274 CHAPTER 7. PROGRAMMING IN R Chapter 8 Reproducible reports with R markdown This chapter will introduce you to creating reproducible reports using R markdown to encourage best (or better) practice to facilitate open science. It will first describe what R markdown is and why you might want to consider using it, describe how to create an R markdown document using RStudio and then how to convert this document to a html or pdf formatted report. During this Chapter you will learn about the different components of an R markdown document, how to format text, graphics and tables within the document and finally how to avoid some of the common difficulties using R markdown. 8.1 What is R markdown? R markdown is a simple and easy to use plain text language used to combine your R code, results from your data analysis (including plots and tables) and written commentary into a single nicely formatted and reproducible document (like a report, publication, thesis chapter or a web page like this one). Technically, R markdown is a variant of another language (yet another language!) called Markdown and both are a type of ‘markup’ language. A markup language simply provides a way of creating an easy to read plain text file which can incorporate formatted text, images, headers and links to other documents. Don’t worry about the details for the moment, although if you’re interested you can find more information about markup languages here. Actually, if it makes you feel any better all of you will have been exposed to a markup language before, as most of the internet content you digest every day is underpinned by a markup language called HTML (Hypertext Markup Language). Anyway, the main point is that R markdown is very easy to learn (much, much easier than HTML) and when used with RStudio it’s ridiculously easy to integrate into your workflow to produce feature rich content (so why wouldn’t you?!). 275 276 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN 8.2 Why use R markdown? During the previous Chapters we talked a lot about conducting your research in a robust and reproducible manner to facilitate open science. In a nutshell, open science is about doing all we can to make our data, methods, results and inferences transparent and available to everyone. Some of the main tenets of open science are described here and include: • Transparency in experimental methodology, observation, collection of data and analytical methods. • Public availability and re-usability of scientific data • Public accessibility and transparency of scientific communication • Using web-based tools to facilitate scientific collaboration By now all of you will (hopefully) be using R to explore and analyse your interesting data. As such, you’re already well along the road to making your analysis more reproducible, transparent and shareable. However, perhaps your current workflow looks something like this: Your data is imported from your favourite spreadsheet software into RStudio (or R), you write your R code to explore and analyse your data, you save plots as external files, copy tables of analysis output and then manually combine all of this and your written prose into a single MS Word document (maybe a paper or thesis chapter). Whilst there is nothing particularly wrong with this approach (and it’s certainly better than using point and click software to analyse your data) there are some limitations: 8.2. WHY USE R MARKDOWN? 277 • It’s not particularly reproducible. Because this workflow separates your R code from the final document there are multiple opportunities for undocumented decisions to be made (which plots did you use? what analysis did/didn’t you include? etc). • It’s inefficient. If you need to go back and change something (create a new plot or update your analysis etc) you will need to create or amend multiple documents increasing the risk of mistakes creeping into your workflow. • It’s difficult to maintain. If your analysis changes you again need to update multiple files and documents. • It can be difficult to decide what to share with others. Do you share all of your code (initial data exploration, model validation etc) or just the code specific to your final document? It’s quite a common (and bad!) practice for researchers to maintain two R scripts, one used for the actual analysis and one to share with the final paper or thesis chapter. This can be both time consuming and confusing and should be avoided. Perhaps a more efficient and robust workflow would look something like this: Your data is imported into RStudio (or R) as before but this time all of the R code you used to analyse your data, produce your plots and your written text (Introduction, Materials and Methods, Discussion etc) is contained within a single R markdown document which is then used (along with your data) to automatically create your final document. This is exactly what R markdown allows you to do. Some of the advantages of using R markdown include: 278 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN • Explicitly links your data with your R code and output creating a fully reproducible workflow. ALL of the R code used to explore, summarise and analyse your data can be included in a single easy to read document. You can decide what to include in your final document (as you will learn below) but all of your R code can be included in the R markdown document. • You can create a wide variety of output formats (pdf, html web pages, MS Word and many others) from a single R markdown document which enhances both collaboration and communication. • Enhances transparency of your research. Your data and R markdown file can be included with your publication or thesis chapter as supplementary material or hosted on a GitHub repository (see the GitHub Chapter). • Increases the efficiency of your workflow. If you need to modify or extend your current analysis you just need to update your R markdown document and these changes will automatically be included in your final document. 8.3 Get started with R markdown To use R markdown you will first need to install the rmarkdown package in RStudio (or in the R console if you’re not using RStudio) and any package dependencies. You can find instructions on how to do this for both Windows and Mac OSX operating systems here. If you would like to create pdf documents (or MS Word documents) from your R markdown file you will also need to install a version of LaTeX on your computer. If you’ve not installed LaTeX before, we recommend that you install TinyTeX. Again, instructions on how to do this can be found here. 8.4 Create an R markdown document Right, time to create your first R markdown document. Within RStudio, click on the menu File -> New File -> R Markdown.... In the pop up window, give the document a ‘Title’ and enter the ‘Author’ information (your name) and select HTML as the default output. We can change all of this later so don’t worry about it for the moment. 8.4. CREATE AN R MARKDOWN DOCUMENT 279 You will notice that when your new R markdown document is created it includes some example R markdown code. Normally you would just highlight and delete everything in the document except the information at the top between the --- delimiters (this is called the YAML header which we will discuss in a bit) and then start writing your own code. However, just for now we will use this document to practice converting R markdown to both html and pdf formats and check everything is working. Once you’ve created your R markdown document it’s good practice to save this file somewhere convenient. You can do this by selecting File -> Save from RStudio menu 286 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN These data were obtained from a mesocosm experiment which aimed to examine the effect of benthic polychaete (Nereis diversicolor) biomass. This is generally a good thing (no more random multiple spaces in your text). If you want your text to start on a new line then you can simply add two blank spaces at the end of the preceding line These data were obtained from a mesocosm experiment which aimed to examine the effect benthic polychaete (Nereis diversicolor) biomass. If you really want multiple spaces within your text then you can use the Non breaking space tag These data were obtained from a mesocosm experiment which aimed to examine the effect benthic polychaete (*Nereis diversicolor*) biomass. These data were obtained from a mesocosm experiment which aimed to examine the effect benthic polychaete (Nereis diversicolor) biomass. Headings You can add headings and subheadings to your R markdown document by using the #symbol at the beginning of the line. You can decrease the size of the headings by simply adding more #symbols. For example # Benthic Biodiversity experiment ## Benthic Biodiversity experiment ### Benthic Biodiversity experiment #### Benthic Biodiversity experiment ##### Benthic Biodiversity experiment ###### Benthic Biodiversity experiment results in headings in decreasing size order Benthic Biodiversity experiment Benthic Biodiversity experiment Benthic Biodiversity experiment Benthic Biodiversity experiment Benthic Biodiversity experiment Comments As you can see above the meaning of the #symbol is different when formatting text in an R markdown document compared to a standard R script (which is used to included a comment - remember?!). You can, however, use a #symbol to comment code inside acode chunk as usual (more about this in a bit). If you want to include a comment in your R markdown document outside a code chunk which won’t be included in the final rendered document then enclose your comment between <!-- and -->. <!-- this is an example of how to format a comment using R markdown. --> 287 288 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN Lists If you want to create a bullet point list of text you can format an unordered list with sub items. Notice that the sub-items need to be indented. -item 1 -item 2 +sub-item 2 +sub-item 3 -item 3 -item 4 • item 1 • item 2 –sub-item 2 –sub-item 3 • item 3 • item 4 If you need an ordered list 1. item 1 2. item 2 +sub-item 2 +sub-item 3 3. item 3 4. item 4 8.5. R MARKDOWN ANATOMY 289 1. item 1 2. item 2 • sub-item 2 • sub-item 3 3. item 3 4. item 4 Images Another useful feature is the ability to embed images and links to web pages (or other documents) into your R markdown document. You can include images into your R markdown document in a number of different ways. Perhaps the simplest method is to use  resulting in: The code above will only work if the image file (Cute_grey_kitten.jpg) is in the right place relative to where you saved your .Rmd file. In the example above the image file is in a sub directory (folder) called images in the directory where we saved our my_first_rmarkdown.Rmd file. You can embed images saved in many different file types but perhaps the most common are .jpg and .png. We think a more flexible way of including images in your document is to use the include_graphics() function from the knitr package as this gives finer control over the alignment and image size (it also works more or less the same with both HTML and pdf output formats). However, to do this you will need to include this R code in a ‘code chunk’ which we haven’t covered yet. Despite this we’ll leave the code here for later reference. This code center aligns the image and scales it to 50% of it’s original size. See ?include_graphics for more options. ```{r, echo=FALSE, fig.align='center', out.width='50%'} library(knitr) include_graphics("images/Cute_grey_kitten.jpg") ``` 290 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN Figure 8.1: Cute grey kitten Links In addition to images you can also include links to webpages or other links in your document. Use the following syntax to create a clickable link to an existing webpage. The link text goes between the square brackets and the URL for the webpage between the round brackets immediately after. 8.5. R MARKDOWN ANATOMY 291 You can include a text for your clickable [link](https://www.worldwildlife.org) which gives you: You can include a text for your clickable link 8.5.3 Code chunks Now to the heart of the matter. To include R code into your R markdown document you simply place your code into a ‘code chunk’. All code chunks start and end with three backticks ```. Note, these are also known as ‘grave accents’ or ‘back quotes’ and are not the same as an apostrophe! On most keyboards you can find the backtick on the same key as tilde (~). ```{r} Any valid R code goes here ``` You can insert a code chunk by either typing the chunk delimiters ```{r} and ``` manually or use the RStudio toolbar (the Insert button) or by clicking on the menu Code -> Insert Chunk. Perhaps an even better way is to get familiar with the keyboard shortcuts Ctrl + Alt + I for Windows and Cmd + Option + I on MacOSX. There are a many things you can do with code chunks: you can produce text output from your analysis, create tables and figures and insert images amongst other things. Within the code chunk you can place rules and arguments between the curly brackets {} that give you control over how your code is interpreted and output is rendered. These are known as chunk options. The only mandatory chunk option is the first argument which specifies which language you’re using (rin our case but other languages are supported). Note, all of your chunk options must be written between the curly brackets on one line with no line breaks. You can also specify an optional code chunk name (or label) which can be useful when trying to debug problems and when performing advanced document rendering. In the following block we name the code chunk summary-stats, create a dataframe (dataf) with two variables xand yand then use the summary() function to display some summary statistics . When we run the code chunk both the R code and the resulting output are displayed in the final document. 292 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN ```{r, summary-stats} x <- 1:10 # create an x variable y <- 10:1 # create a y variable dataf <- data.frame(x = x, y = y) summary(dataf) ``` x<- 1:10 # create an x variable y<- 10:1# create a y variable dataf <- data.frame(x = x, y = y) summary(dataf) ## x y ## Min. : 1.00 Min. : 1.00 ## 1st Qu.: 3.25 1st Qu.: 3.25 ## Median : 5.50 Median : 5.50 ## Mean : 5.50 Mean : 5.50 ## 3rd Qu.: 7.75 3rd Qu.: 7.75 ## Max. :10.00 Max. :10.00 When using chunk names make sure that you don’t have duplicate chunk names in your R markdown document and avoid spaces and full stops as this may cause problems when you come to knit your document (We use a -to separate words in our chunk names). If we wanted to only display the output of our R code (just the summary statistics for example) and not the code itself in our final document we can use the chunk option echo=FALSE ```{r, summary-stats, echo=FALSE} x <- 1:10 # create an x variable y <- 10:1 # create a y variable dataf <- data.frame(x = x, y = y) summary(dataf) ``` 8.5. R MARKDOWN ANATOMY 293 ## x y ## Min. : 1.00 Min. : 1.00 ## 1st Qu.: 3.25 1st Qu.: 3.25 ## Median : 5.50 Median : 5.50 ## Mean : 5.50 Mean : 5.50 ## 3rd Qu.: 7.75 3rd Qu.: 7.75 ## Max. :10.00 Max. :10.00 To display the R code but not the output use the results='hide' chunk option. ```{r, summary-stats, results='hide'} x <- 1:10 # create an x variable y <- 10:1 # create a y variable dataf <- data.frame(x = x, y = y) summary(dataf) ``` x<- 1:10 # create an x variable y<- 10:1# create a y variable dataf <- data.frame(x = x, y = y) summary(dataf) Sometimes you may want to execute a code chunk without showing any output at all. You can suppress the entire output using the chunk option include=FALSE. ```{r, summary-stats, include=FALSE} x <- 1:10 # create an x variable y <- 10:1 # create a y variable dataf <- data.frame(x = x, y = y) summary(dataf) ``` There are a large number of chunk options documented here with a more condensed version here. Perhaps the most commonly used are summarised below with the default values shown. 294 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN Chunk option default value Function echo echo=TRUE If FALSE, will not display the code in the final document results results='markup' If ‘hide’, will not display the code’s results in the final document. If ‘hold’, will delay displaying all output pieces until the end of the chunk. If ‘asis’, will pass through results without reformatting them. include include=TRUE If FALSE, will run the chunk but not include the chunk in the final document. eval eval=TRUE If FALSE, will not run the code in the code chunk. message message=TRUE If FALSE, will not display any messages generated by the code. warning warning=TRUE If FALSE, will not display any warning messages generated by the code. 8.5.4 Adding figures By default, figures produced by R code will be placed immediately after the code chunk they were generated from. For example: ```{r, simple-plot} x <- 1:10 # create an x variable y <- 10:1 # create a y variable dataf <- data.frame(x = x, y = y) plot(dataf$x, dataf$y, xlab = "x axis", ylab = "y axis") ``` 8.5. R MARKDOWN ANATOMY 295 x<- 1:10 # create an x variable y<- 10:1# create a y variable dataf <- data.frame(x = x, y = y) plot(dataf$x, dataf$y, xlab = "x axis",ylab = "y axis") 2 4 6 8 10 2 4 6 8 10 x axis y axis If you want to change the plot dimensions in the final document you can use the fig.width= and fig.height= chunk options (in inches!). You can also change the alignment of the figure using the fig.align= chunk option. ```{r, simple-plot, fig.width=4, fig.height=3, fig.align='center'} x <- 1:10 # create an x variable y <- 10:1 # create a y variable dataf <- data.frame(x = x, y = y) plot(dataf$x, dataf$y, xlab = "x axis", ylab = "y axis") ``` x<- 1:10 # create an x variable y<- 10:1# create a y variable dataf <- data.frame(x = x, y = y) plot(dataf$x, dataf$y, xlab = "x axis",ylab = "y axis") 302 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN If you want even more control and customisation options for your tables take a look at the pander and xtable packages. 8.5.6 Inline R code Up till now we’ve been writing and executing our R code in code chunks. Another great reason to use R markdown is that we can also include our R code directly within our text. This is known as ‘inline code’. To include your code in your R markdown text you simply write `r write your code here`. This can come in really useful when you want to include summary statistics within your text. For example, we could describe the iris dataset as follows: Morphological characteristics (variable names: `r names(iris)[1:4]`) were measured from `r nrow(iris)` *Iris sp.* plants from `r length(levels(iris$Species))` different species. The mean Sepal length was `r round(mean(iris$Sepal.Length), digits = 2)` mm. which will be rendered as Morphological characteristics (variable names: Sepal.Length, Sepal.Width, Petal.Length, Petal.Width) were measured from 150 iris plants from 3 different species. The mean Sepal length was 5.84 mm. The great thing about including inline R code in your text is that these values will automatically be updated if your data changes. 8.6 Some tips and tricks Problem : When rendering my R markdown document to pdf my code runs off the edge of the page. Solution: Add a global_options argument at the start of your .Rmd file in a code chunk: ```{r, global_options, include=FALSE} knitr::opts_chunk$set(message=FALSE, tidy.opts=list(width.cutoff=60), tidy=TRUE) ``` 8.6. SOME TIPS AND TRICKS 303 This code chunk won’t be displayed in the final document due to the include = FALSE argument and you should place the code chunk immediately after the YAML header to affect everything below that. tidy.opts = list(width.cutoff = 60), tidy=TRUE defines the margin cutoff point and wraps text to the next line. Play around with this value to get it right (60-80 should be OK for most documents). Problem: When I load a package in my R markdown document my rendered output contains all of the startup messages and/or warnings. Solution: You can load all of your packages at the start of your R markdown document in a code chunk along with setting your global options. ```{r, global_options, include=FALSE} knitr::opts_chunk$set(message=FALSE, warning=FALSE, tidy.opts=list(width.cutoff=60)) suppressPackageStartupMessages(library(ggplot2)) ``` The message=FALSE and warning=FALSE arguments suppress messages and warnings. The suppressPackageStartupMessages(library(ggplot2)) will load the ggplot2 package but suppress startup messages. Problem: When rendering my R markdown document to pdf my tables and/or figures are split over two pages. Solution: Add a page break using the LateX \pagebreak notation before your offending table or figure. Problem: The code in my rendered document looks ugly! Solution: Add the argument tidy=TRUE to your global arguments. Sometimes, however, this can cause problems especially with correct code indentation. 304 CHAPTER 8. REPRODUCIBLE REPORTS WITH R MARKDOWN ```{r, global_options, include=FALSE} knitr::opts_chunk$set(message=FALSE, tidy.opts=list(width.cutoff=60), tidy=TRUE) ``` 8.7 Further Information Although we’ve covered more than enough to get you quite far using R markdown, as with most things R related, we’ve really only had time to scratch the surface. Happily, there’s a wealth of information available to you should you need to expand your knowledge and experience. A good place to start is the excellent free book written by the creator of R markdown Yihui Xie which you can find here. Another useful and concise R markdown reference guide can be found here. A quick and easy RStudio R markdown cheatsheet. Chapter 9 Version control with Git and GitHub This Chapter will introduce you to the basics of using a version control system to keep track of all your important R code and facilitate collaboration with colleagues and the wider world. This Chapter will focus on using the software ‘Git’ in combination with the web-based hosting service ‘GitHub’. By the end of the Chapter, you will be able to install and configure Git and GitHub on your computer and setup and work with a version controlled project in RStudio. We won’t be covering more advanced topics such as branching, forking and pull requests in much detail but we do give an overview later on in the Chapter. Just a few notes of caution. In this Chapter we’ll be using RStudio to interface with Git as it gives you a nice friendly graphical user interface which generally makes life a little bit easier (and who doesn’t want that?). However, one downside to using RStudio with Git is that RStudio only provides pretty basic Git functionality through its menu system. That’s fine for most of what we’ll be doing during this Chapter (although we will introduce a few Git commands as we go along) but if you really want to benefit from using Git’s power you will need to learn some Git commands and syntax. This leads us on to our next point. We’re not going to lie, Git can become a little bewildering and frustrating when you first start using it. This is mostly due to the terminology and liberal use of jargon associated with Git, but there’s no hiding the fact that it’s quite easy to get yourself and your Git repository into a pickle. Therefore, we’ve tried hard to keep things as straight forward as we can during this Chapter and as a result we do occasionally show you a couple of very ‘un-Git’ ways of doing things (mostly about reverting to previous versions of documents). Don’t get hung up about this, there’s no shame to using these low tech solutions and if it works then it works. Lastly, GitHub was not designed to host very large files and will warn you if you try to add files greater than 50 MB and block you adding files greater than 100 MB. If your project involves using large file sizes there are a few solutions but we find the easiest solution is to host these files elsewhere (Googledrive, Dropbox etc) and create a link to them in a README file or R markdown document on Github. 305 306 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB 9.1 What is version control? AVersion Control System (VCS) keeps a record of all the changes you make to your files that make up a particular project and allows you to revert to previous versions of files if you need to. To put it another way, if you muck things up or accidentally lose important files you can easily roll back to a previous stage in your project to sort things out. Version control was originally designed for collaborative software development, but it’s equally useful for scientific research and collaborations (although admittedly a lot of the terms, jargon and functionality are focused on the software development side). There are many different version control systems currently available, but we’ll focus on using Git, because it’s free and open source and it integrates nicely with RStudio. This means that its can easily become part of your usual workflow with minimal additional overhead. 9.2 Why use version control? So why should you worry about version control? Well, first of all it helps avoid this (familiar?) situation when you’re working on a project Figure 9.1: You need version control usually arising from this (familiar?) scenario 9.3. WHAT IS GIT AND GITHUB? 307 Version control automatically takes care of keeping a record of all the versions of a particular file and allows you to revert back to previous versions if you need to. Version control also helps you (especially the future you) keep track of all your files in a single place and it helps others (especially collaborators) review, contribute to and reuse your work through the GitHub website. Lastly, your files are always available from anywhere and on any computer, all you need is an internet connection. 9.3 What is Git and GitHub? Git is a version control system originally developed by Linus Torvalds that lets you track changes to a set of files. These files can be any type of file including the menagerie of files that typically make up a data orientated project (.pdf, .Rmd, .docx, .txt, .jpg etc) although plain text files work the best. All the files that make up a project is called arepository (or just repo). GitHub is a web-based hosting service for Git repositories which allows you to create a remote copy of your local version-controlled project. This can be used as a backup or archive of your project or make it accessible to you and to your colleagues so you can work collaboratively. 308 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB At the start of a project we typically (but not always) create a remote repository on GitHub, then clone (think of this as copying) this repository to our local computer (the one in front of you). This cloning is usually a one time event and you shouldn’t need to clone this repository again unless you really muck things up. Once you have cloned your repository you can then work locally on your project as usual, creating and saving files for your data analysis (scripts, R markdown documents, figures etc). Along the way you can take snapshots (called commits) of these files after you’ve made important changes. We can then push these changes to the remote GitHub repository to make a backup or make available to our collaborators. If other people are working on the same project (repository), or maybe you’re working on a different computer, you can pull any changes back to your local repository so everything is synchronised. 9.4 Getting started This Chapter assumes that you have already installed the latest versions of R and RStudio. If you haven’t done this yet you can find instructions here. 9.4.1 Install Git To get started, you first need to install Git. If you’re lucky you may already have Git installed (especially if you have a Mac or Linux computer). You can check if you already have Git installed by clicking on the Terminal tab in the Console window in RStudio and typing git --version (the space after the git command is important). If you see something that looks like git version 2.25.0 (the version number may be different on your computer) then you already have Git installed (happy days). If you get an error (something like git: command not found) this means you don’t have Git installed (yet!). 9.4. GETTING STARTED 309 You can also do this check outside RStudio by opening up a separate Terminal if you want. On Windows go to the ‘Start menu’ and in the search bar (or run box) type cmd and press enter. On a Mac go to ‘Applications’ in Finder, click on the ‘Utilities’ folder and then on the ‘Terminal’ program. On a Linux machine simply open the Terminal (Ctrl+Alt+T often does it). To install Git on a Windows computer we recommend you download and install Git for Windows (also known as ‘Git Bash’). You can find the download file and installation instructions here. For those of you using a Mac computer we recommend you download Git from here and install in the usual way (double click on the installer package once downloaded). If you’ve previously installed Xcode on your Mac and want to use a more up to date version of Git then you will need to follow a few more steps documented here. If you’ve never heard of Xcode then don’t worry about it! For those of you lucky enough to be working on a Linux machine you can simply use your OS package manager to install Git from the official repository. For Ubuntu Linux (or variants of) open your Terminal and type sudo apt update sudo apt install git You will need administrative privileges to do this. For other versions of Linux see here for further installation instructions. Whatever version of Git you’re installing, once the installation has finished verify that the installation process has been successful by running the command git --version in the Terminal tab in RStudio (as described above). On some installations of Git (yes we’re looking at you MS Windows) this may still produce an error as you will also need to setup RStudio so it can find the Git executable (described below). 9.4.2 Configure Git After installing Git, you need to configure it so you can use it. Click on the Terminal tab in the Console window again and type the following: 310 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB git config --global user.email '[email protected]' git config --global user.name 'Your Name' substituting 'Your Name' for your actual name and '[email protected]' with your email address. We recommend you use your University email address (if you have one) as you will also use this address when you register for your GitHub account (coming up in a bit). If this was successful, you should see no error messages from these commands. To verify that you have successfully configured Git type the following into the Terminal git config --global --list You should see both your user.name and user.email configured. 9.4.3 Configure RStudio As you can see above, Git can be used from the command line, but it also integrates well with RStudio, providing a friendly graphical user interface. If you want to use RStudio’s Git integration (we recommend you do - at least at the start), you need to check that the path to the Git executable is specified correctly. In RStudio, go to the menu Tools -> Global Options -> Git/SVN and make sure that ‘Enable version control interface for RStudio projects’ is ticked and that the ‘Git executable:’ path is correct for your installation. If it’s not correct hit the Browse... button and navigate to where you installed git and click on the executable file. You will need to restart RStudio after doing this. 9.5. SETTING UP A PROJECT IN RSTUDIO 311 9.4.4 Register a GitHub account If all you want to do is to keep track of files and file versions on your local computer then Git is sufficient. If however, you would like to make an off-site copy of your project or make it available to your collaborators then you will need a web-based hosting service for your Git repositories. This is where GitHub comes into play (there are also other services like GitLab,Bitbucket and Savannah). You can sign up for a free account on GitHub here. You will need to specify a username, an email address and a strong password. We suggest that you use your University email address (if you have one) as this will also allow you to apply for a free educator or researcher account later on which gives you some useful benefits (don’t worry about this now though). When it comes to choosing a username we suggest you give this some thought. Choose a short(ish) rather than a long username, use all lowercase and hyphenate if you want to include multiple words, find a way of incorporating your actual name and lastly, choose a username that you will feel comfortable revealing to your future employer! Next click on the ‘Select a plan’ (you may have to solve a simple puzzle first to verify you’re human) and choose the ‘Free Plan’ option. Github will send an email to the email address you supplied for you to verify. Once you’ve completed all those steps you should have both Git and GitHub setup up ready for you to use (Finally!). 9.5 Setting up a project in RStudio Now that you’re all set up, let’s create your first version controlled RStudio project. There are a couple of different approaches you can use to do this. You can either setup a remote GitHub repository first then connect an RStudio project to this repository (we’ll call this Option 1). Another option is to setup a local repository first and then link a remote GitHub repository to this repository (Option 2). You can also connect an existing project to a GitHub repository but we won’t cover this here. We suggest that if you’re completely new to Git and GitHub then use Option 1 as this approach sets up your local Git repository nicely and you can push and pull immediately. Option 2 requires a little more work and therefore there are more opportunities to go wrong. We will cover both of these options below. 9.5.1 Option 1 - GitHub first To use the GitHub first approach you will first need to create a repository (repo) on GitHub. Go to your GitHub page and sign in if necessary. Click on the ‘Repositories’ tab at the top and then on the green ‘New’ button on the right 318 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB You can see that both files are listed. Next, tick the boxes under the ‘Staged’ column for both files and then click on the ‘Commit’ button. 9.5. SETTING UP A PROJECT IN RSTUDIO 319 This will take you to the ‘Review Changes’ window. Type in the commit message ‘First commit’ in the ‘Commit message’ window and click on the ‘Commit’ button. A new window will appear with some messages which you can ignore for now. Click ‘Close’ to close this window and also close the ‘Review Changes’ window. The two files should now have disappeared from the Git pane in RStudio indicating a successful commit. OK, that’s those two files now under version control. Now we need to create a new repository on GitHub. In your browser go to your GitHub page and sign in if necessary. Click on the ‘Repositories’ tab and then click on the green ‘New’ button on the right. Give your new repo the name second_repo (the same as your version controlled directory name) and select ‘Public’. This time do not tick the ‘Initialize this repository with a README’ (this is important) and then click on ‘Create repository’. 320 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB This will take you to a Quick setup page which provides you with some code for various situations. The code we are interested in is the code under ...or push an existing repository from the command line heading. Highlight and copy the first line of code (note: yours will be slightly different as it will include your GitHub username not mine) git remote add origin https://github.com/alexd106/second_repo.git Switch to RStudio, click on the ‘Terminal’ tab and paste the command into the Terminal. Now go back to GitHub and copy the second line of code git push -u origin master and paste this into the Terminal in RStudio. You should see something like this If you take a look at your repo back on GitHub (click on the /second_repo link at the top) you will see the second_repo.Rproj and .gitignore files have now been pushed to GitHub from your local repository. 9.5. SETTING UP A PROJECT IN RSTUDIO 321 The last thing we need to do is create and add a README file to your repository. A README file describes your project and is written using the same Markdown language you learned in the R markdown Chapter. A good README file makes it easy for others (or the future you!) to use your code and reproduce your project. You can create a README file in RStudio or in GitHub. Let’s use the second option. In your repository on GitHub click on the green Add a README button. Now write a short description of your project in the <> Edit new file section and then click on the green Commit new file button. 322 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB You should now see the README.md file listed in your repository. It won’t actually exist on your computer yet as you will need to pull these changes back to your local repository, but more about that in the next section. Whether you followed Option 1 or Option 2 (or both) you have now successfully setup a version controlled RStudio project (and associated directory) and linked this to a GitHub repository. Git will now monitor this directory for any changes you make to files and also if you add or delete files. If the steps above seem like a bit of an ordeal, just remember, you only need to do this once for each project and it gets much easier over time. 9.6 Using Git Now that we have our project and repositories (both local and remote) set up, it’s finally time to learn how to use Git in RStudio! Typically, when using Git your workflow will go something like this: 1. You create/delete and edit files in your project directory on your computer as usual (saving these changes as you go) 2. Once you’ve reached a natural ‘break point’ in your progress (i.e. you’d be sad if you lost this progress) you stage these files 9.6. USING GIT 323 3. You then commit the changes you made to these staged files (along with a useful commit message) which creates a permanent snapshot of these changes 4. You keep on with this cycle until you get to a point when you would like to push these changes to GitHub 5. If you’re working with other people on the same project you may also need to pull their changes to your local computer OK, let’s go through an example to help clarify this workflow. In RStudio open up the first_repo.Rproj you created previously during Option 1. Either use the File -> Open Project menu or click on the top right project icon and select the appropriate project. Create an R markdown document inside this project by clicking on the File -> New File -> R markdown menu (remember from the R markdown Chapter?). Once created, we can delete all the example R markdown code (except the YAML header) as usual and write some interesting R markdown text and include a plot. We’ll use the inbuilt cars dataset to do this. Save this file (cmd + s for Mac or ctrl + s in Windows). Your R markdown document should look something like the following (it doesn’t matter if it’s not exactly the same). 324 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB Take a look at the ‘Git’ tab which should list your new R markdown document (first_doc.Rmd in this example) along with first_repo.Rproj, and .gitignore (you created these files previously when following Option 1). Following our workflow, we now need to stage these files. To do this tick the boxes under the ‘Staged’ column for all files. Notice that there is a status icon next to the box which gives you an indication of how the files were changed. In our case all of the files are to be added (capital A) as we have just created them. RStudio IDE : : CHEAT SHEET Write Code Pro Features RStudio® is a trademark of RStudio, Inc. • CC BY SA RStudio • [email protected] • 844-448-1212 • rstudio.com • Learn more at www.rstudio.com • RStudio IDE 0.99.832 • Updated: 2016-01 Turn project into package, Enable roxygen documentation with Tools > Project Options > Build Tools Roxygen guide at Help > Roxygen Quick Reference File > New Project > New Directory > R Package Share Project with Collaborators Active shared collaborators Select R Version Start new R Session in current project Close R Session in project J H T RStudio saves the call history, workspace, and working directory associated with a project. It reloads each when you re-open a project. Name of current project View(<data>) opens spreadsheet like view of data set Sort by values Filter rows by value or value range Search for value Viewer Pane displays HTML content, such as Shiny apps, RMarkdown reports, and interactive visualizations Stop Shiny app Publish to shinyapps.io, rpubs, RSConnect, … Refresh RStudio opens documentation in a dedicated Help pane Home page of helpful links Search within help file Search for help file GUI Package manager lists every installed package Click to load package with library(). Unclick to detach package with detach() Delete from library Install Packages Update Packages Create reproducible package library for your project RStudio opens plots in a dedicated Plots pane Navigate recent plots Open in window Export plot Delete plot Delete all plots Package version installed Examine variables in executing environment Open with debug(), browser(), or a breakpoint. RStudio will open the debugger mode when it encounters a breakpoint while executing code. Open traceback to examine the functions that R called before the error occurred Launch debugger mode from origin of error Click next to line number to add/remove a breakpoint. Select function in traceback to debug Highlighted line shows where execution has paused Run commands in environment where execution has paused Step through code one line at a time Step into and out of functions to run Resume execution Quit debug mode Open Shiny, R Markdown, knitr, Sweave, LaTeX, .Rd files and more in Source Pane Check spelling Render output Choose output format Choose output location Insert code chunk Jump to previous chunk Jump to next chunk Run selected lines Publish to server Show file outline Set knitr chunk options Run this and all previous code chunks Run this code chunk Jump to chunk RStudio recognizes that files named app.R, server.R, ui.R, and global.R belong to a shiny app Run app Choose location to view app Publish to shinyapps.io or server Manage publish accounts Access markdown guide at Help > Markdown Quick Reference Stage files: Show file diff Commit staged files Push/Pull to remote View History current branch • Added • Deleted • Modified • Renamed • Untracked Turn on at Tools > Project Options > Git/SVN Open shell to type commands A D M R ? Search inside environment Syntax highlighting based on your file's extension Code diagnostics that appear in the margin. Hover over diagnostic symbols for details. Tab completion to finish function names, file paths, arguments, and more. Multi-language code snippets to quickly use common blocks of code. Open in new window Save Find and replace Compile as notebook Run selected code Re-run previous code Source with or without Echo Show file outline Jump to function in file Change file type Navigate tabs A File browser keyed to your working directory. Click on file or directory name to open. Path to displayed directory Upload file Create folder Delete file Rename file Change directory Displays saved objects by type with short description View function source code View in data viewer Load workspace Save workspace Import data with wizard Delete all saved objects Display objects as list or grid Choose environment to display from list of parent environments History of past commands to run/copy Display .RPres slideshows File > New File > R Presentation Working Directory Maximize, minimize panes Drag pane boundaries J H T Cursors of shared users File > New Project Press ! to see command history Multiple cursors/column selection with Alt + mouse drag. Documents and Apps R Support PROJECT SYSTEM Debug Mode Version Control with Git or SVN Package Writing 9.6. USING GIT 325 After you have staged the files the next step is to commit the files. This is done by clicking on the ‘Commit’ button. After clicking on the ‘Commit’ button you will be taken to the ‘Review Changes’ window. You should see the three files you staged from the previous step in the left pane. If you click on the file name first_doc.Rmd you will see the changes you have made to this file highlighted in the bottom pane. Any content that you have added is highlighted in green and deleted content is highlighted in red. As you have only just created this file, all the content is highlighted in green. To commit these files (take a snapshot) first enter a mandatory commit message in the ‘Commit message’ box. This message should be relatively short and informative (to you and your collaborators) and indicate why you made the changes, not what you changed. This makes sense as Git keeps track of what has changed and so it is best not to use commit messages for this purpose. It’s traditional to enter the message ‘First commit’ (or ‘Initial commit’) when you commit files for the first time. Now click on the ‘Commit’ button to commit these changes. 326 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB A summary of the commit you just performed will be shown. Now click on the ‘Close’ button to return to the ‘Review Changes’ window. Note that the staged files have now been removed. Now that you have committed your changes the next step is to push these changes to GitHub. Before you push your changes it’s good practice to first pull any changes from GitHub. This is especially important if both you and your collaborators are working on the same files as it keeps you local copy up to date and avoids any potential conflicts. In this case your repository will already be up to date but it’s a good habit to get into. To do this, click on the ‘Pull’ button on the top right of the ‘Review Changes’ window. Once you have pulled any changes click on the green ‘Push’ button to push your changes. You will see a summary of the push you just performed. Hit the ‘Close’ button and then close the ‘Review Changes’ window. 9.6. USING GIT 327 To confirm the changes you made to the project have been pushed to GitHub, open your GitHub page, click on the Repositories link and then click on the first_repo repository. You should see four files listed including the first_doc.Rmd you just pushed. Along side the file name you will see your last commit message (‘First commit’ in this case) and when you made the last commit. To see the contents of the file click on the first_doc.Rmd file name. 9.6.1 Tracking changes After following the steps outlined above, you will have successfully modified an RStudio project by creating a new R markdown document, staged and then committed these 334 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB 9.6.3 Reverting changes One the great things about using Git is that you are able to revert to previous versions of files if you’ve made a mistake, broke something or just prefer and earlier approach. How you do this will depend on whether the changes you want to discard have been staged, committed or pushed to GitHub. We’ll go through some common scenarios below mostly using RStudio but occasionally we will need to resort to using the Terminal (still in RStudio though). Changes saved but not staged, committed or pushed If you have saved changes to your file(s) but not staged, committed or pushed these files to GitHub you can right click on the offending file in the Git pane and select ‘Revert …’. This will roll back all of the changes you have made to the same state as your last commit. Just be aware that you cannot undo this operation so use with caution. 9.6. USING GIT 335 You can also undo changes to just part of a file by opening up the ‘Diff’ window (click on the ‘Diff’ button in the Git pane). Select the line you wish to discard by double clicking on the line and then click on the ‘Discard line’ button. In a similar fashion you can discard chunks of code by clicking on the ‘Discard chunk’ button. Staged but not committed and not pushed If you have staged your files, but not committed them then simply unstage them by clicking on the ‘Staged’ check box in the Git pane (or in the ‘Review Changes’ window) to remove the tick. You can then revert all or parts of the file as described in the section above. Staged and committed but not pushed If you have made a mistake or have forgotten to include a file in your last commit which you have not yet pushed to GitHub, you can just fix your mistake, save your changes, and then amend your previous commit. You can do this by staging your file and then tick the ‘Amend previous commit’ box in the ‘Review Changes’ window before committing. 336 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB If we check out our commit history you can see that our latest commit contains both changes to the file rather than having two separate commits. We use the amend commit approach alot but it’s important to understand that you should not do this if you have already pushed your last commit to GitHub as you are effectively rewriting history and all sorts bad things may happen! 9.6. USING GIT 337 If you spot a mistake that has happened multiple commits back or you just want to revert to a previous version of a document you have a number of options. Option 1 - (probably the easiest but very unGit - but like, whatever!) is to look in your commit history in RStudio, find the commit that you would like to go back to and click on the ‘View file @’ button to show the file contents. 338 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB You can then copy the contents of the file to the clipboard and paste it into your current file to replace your duff code or text. Alternatively, you can click on the ‘Save As’ button and save the file with a different file name. Once you have saved your new file you can delete your current unwanted file and then carry on working on your new file. Don’t forget to stage and commit this new file. 9.6. USING GIT 339 Option 2 - (Git like) Go to your Git history, find the commit you would like to roll back to and write down (or copy) its SHA identifier. Now go to the Terminal in RStudio and type git checkout <SHA> <filename>. In our case the SHA key is 2b4693d1 and the filename is first_doc.Rmd so our command would look like this: git checkout 2b4693d1 first_doc.Rmd The command above will copy the selected file version from the past and place it into the present. RStudio may ask you whether you want to reload the file as it now changed - select yes. You will also need to stage and commit the file as usual. If you want to revert all your files to the same state as a previous commit rather than just one file you can use (the single ‘dot’ .is important otherwise your HEAD will detach!): 340 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB git rm -r . git checkout 2b4693d1 . Note that this will delete all files that you have created since you made this commit so be careful! Staged, committed and pushed If you have already pushed your commits to GitHub you can use the git checkout strategy described above and then commit and push to update GitHub (although this is not really considered ‘best’ practice). Another approach would be to use git revert (Note: as far as we can tell git revert is not the same as the ‘Revert’ option in RStudio). The revert command in Git essentially creates a new commit based on a previous commit and therefore preserves all of your commit history. To rollback to a previous state (commit) you first need to identify the SHA for the commit you wish to go back to (as we did above) and then use the revert command in the Terminal. Let’s say we want to revert back to our ‘First commit’ which has a SHA identifier d27e79f1. We can use the revert command as shown below in the Terminal. The --no-commit option is used to prevent us from having to deal with each intermediate commit. git revert --no-commit d27e79f1..HEAD Your first_doc.Rmd file will now revert back to the same state as it was when you did your ‘First commit’. Notice also that the first_doc.pdf file has been deleted as this wasn’t present when we made our first commit. You can now stage and commit these files with a new commit message and finally push them to GitHub. Notice that if we look at our commit history all of the commits we have made are still present. 9.6. USING GIT 341 and our repo on GitHub also reflects these changes 342 CHAPTER 9. VERSION CONTROL WITH GIT AND GITHUB 9.6.4 Collaborate with Git GitHub is a great tool for collaboration, it can seem scary and complicated at first, but it is worth investing some time to learn how it works. What makes GitHub so good for collaboration is that it is a distributed system, which means that every collaborator works on their own copy of the project and changes are then merged together in the remote repository. There are two main ways you can set up a collaborative project on GitHub. One is the workflow we went through above, where everybody connects their local repository to the same remote one; this system works well with small projects where different people mainly work on different aspects of the project but can quickly become unwieldy if many people are collaborating and are working on the same files (merge misery!). The second approach consists of every collaborator creating a copy (or fork) of the main repository, which becomes their remote repository. Every collaborator then needs to send a request (a pull request) to the owner of the main repository to incorporate any changes into the main repository and this includes a review process before the changes are integrated. More detail of these topics can be found in the Further resources section. 9.6.5 Git tips Generally speaking you should commit often (including amended commits) but push much less often. This makes collaboration easier and also makes the process of reverting to previous versions of documents much more straight forward. We generally only push changes to GitHub when we’re happy for our collaborators (or the rest of the world) to see our work. However, this is entirely up to you and depends on the project (and who you are working with) and what your priorities are when using Git. If you don’t want to track a file in your repository (maybe they are too large or transient files) you can get Git to ignore the file by right clicking on the filename in the Git pane and selecting ‘Ignore…’ This will add the filename to the .gitignore file. If you want to ignore multiple files or a particular type of file you can also include wildcards in the .gitignore file. For example 9.7. FURTHER RESOURCES 343 to ignore all png files you can include the expression *.png in your .gitignore file and save. If it all goes pear shaped and you end up completely trashing your Git repository don’t despair (we’ve all been there!). As long as your GitHub repository is good, all you need to do is delete the offending project directory on your computer, create a new RStudio project and link this with your remote GitHub repository using Option 2. Once you have cloned the remote repository you should be good to go. 9.7 Further resources There are many good online guides to learn more about git and GitHub and as with any open source software there is a huge community that can be a great resource: • The British Ecological Society guide to Reproducible Code • The GitHub guides • The Mozilla Science Lab GitHub for Collaboration on Open Projects guide • Jenny Bryan’s Happy Git and GitHub. We borrowed the idea (but with different content) of RStudio first, RStudio second in the ‘Setting up a version controlled Project in RStudio’ section. • Melanie Frazier’s GitHub: A beginner’s guide to going back in time (aka fixing mistakes). We followed this structure (with modifications and different content) in the ‘Reverting changes’ section. • If you have done something terribly wrong and don’t know how to fix it try Oh Shit, Git or if you’re easily offended Dangit, Git These are only a couple of examples, all you need to do is Google “version control with git and GitHub” to see how huge the community around these open source projects is and how many free resources are available for you to become a version control expert. 350 INDEX geom_quantile(), 222 geom_smooth(), 163,188,193 geom_text(), 208,210 geom_violin(), 220 getwd(), 29 GGally package, 227 ggfortify package, 251 ggpairs(), 227 ggplot(), 158,166,174,189,241 ggplot2 package, 155,256 ggsave(), 192 ggthemes package, 183 glm(), 254 gls(), 254 grid(), 149 help(), 57 help.search(), 59 here(), 31 hexbin package, 224 hist(), 116 histogram(), 132 identical(), 260 if(), 262 ifelse(), 263 include_graphics(), 289 install.packages(), 23 install_github(), 24 is.character(), 64 is.factor(), 65 is.logical(), 64 is.na(), 264 is.numeric(), 64,65 jpeg(), 154 kable(), 300,328 kableExtra package, 301 knitr package, 289,328 krustkal.test(), 254 labs(), 179,209 lapply(), 271 lattice package, 132 layout(), 151 layout.show(), 151 legend(), 148 length(), 48 library(), 24,297 lines(), 118,149,247 list(), 68,105 list.files(), 33 lm(), 239 lme(), 254 lmer(), 254 load(), 62 log(), 42 log10(), 42 martix(), 66 matrix(), 151,236 mean(), 48,104,105,201 melt(), 99 merge(), 97 mtext(), 149 names(), 69 ncol(), 269 order(), 54,89 pairs(), 125 panel.cor(), 126 panel.hist(), 127 panel.smooth(), 128 par(), 139,141,144 par(mfrow=), 149 paste(), 138 patchwork package, 171 pdf(), 153 plot(), 113,137,145,251 plot_layout(), 213 png(), 154 points(), 145 polygon(), 149 predict(), 247 prop.test(), 235 qqline(), 232 qqnorm(), 232 rbind(), 94 RColorBrewer package, 142 read.csv(), 77 read.csv2(), 77 read.delim(), 77 read.table package, 79 INDEX 351 read.table(), 75,241 read_csv(), 80 read_delim(), 80 read_table(), 80 read_tsv(), 80 readr package, 79 rect(), 149 remotes package, 23 render(), 282 rep(), 48 reshape2 package, 99 resid(), 248 return(), 259 rev(), 54 rmarkdown package, 278 rnorm(), 257 rownames(), 67,226 RSiteSearch(), 60 rstandard(), 253 rstudent(), 253 sapply(), 271 save(), 61 save.image(), 62 scale_colour_gradient(), 199 scale_colour_gradient2(), 199 scale_colour_gradientn(), 202 scale_colour_manual(), 189 scale_shape_manual(), 189 scale_x_continuous(), 210 scale_x_discrete(), 210 scale_y_continuous(), 210 scale_y_discrete(), 210 scale_y_sqrt(), 220 sd(), 48,104 seq(), 48,117,210 session_info(), 35 sessionInfo(), 35 setwd(), 30 shaprio.test(), 232 sort, 53 splom(), 133 sqrt(), 42 str(), 71,243 subset(), 88 summary(), 81,101,104,244,246,291, 328 t(), 67 t.test(), 229 table(), 102 tapply(), 104,271 text(), 143 theme(), 205,211 theme_bw(), 181 theme_classic(), 181,183 theme_light(), 181 theme_minimal(), 181 theme_rbook(), 184,204,256 tiff(), 154 update(), 252 update.packages(), 23 var(), 48 var.test(), 234 vioplot package, 123 vioplot(), 123 warning(), 265 wilcox.test(), 229 windowsFonts(), 208 with(), 105,114 write.csv(), 109 write.table(), 108 write_csv(), 110 write_excel_csv(), 110 write_tsv(), 110 xlab(), 177 xlim(), 195 xtabs(), 103 xtfrm, 92 xyplot(), 133,135 ylab(), 177 ylim(), 195