• We rely on tidyverse for data manipulation
• tidyverse is an ecosystem of packages
• Has dplyr package, which is dedicated for data manipulation
• tidyverse does not load by default, we need to load it
• It brings dplyr and other needed packages

1. Each variable has its own column.
2. Each observation has its own row.
3. Each value has its own cell.

temperature = tibble(
  year = 2019:2023,
  dodoma = rnorm(n = 5, mean = 21, sd = .2),
  Dar = rnorm(n = 5, mean = 28, sd = 1.2),
  Mwanza = rnorm(n = 5, mean = 21, sd = .6),
  Tanga = rnorm(n = 5, mean = 26, sd = 1.8),
) |> 
  mutate(across(is.numeric, round, 2))

Wide format

Long format

• dplyr has set of functions for data manipulation
• These functions are called verbs
• Similar to verbs in English language,
• dplyr is also called grammar of data manipulation

• filter
• slice
• select
• mutate
• arrange
• summarise
• group_by

• Each verb does one particular computation
• Require combination of multiple verbs
• Chain the process with pipe operator (%>%)

• the rename() streamline renaming data frame columns.
• Efficient for renaming multiple columns
• The rename() simplifies data wrangling tasks
• enhance code readability and efficiency

• filter() used to subset data frames based on conditions.
• Uses logical (comparison) operations (<, <=, >, >=, &, |, ==)
• Only records that meet the condition (valid) are displayed
• If the condition does not meet condition (invalid) throw an error

• For single variable

lfq_clean |> 
  filter(
    species_fish == "Lethrinus mahsena"
    )

• Chain for more than one variable

lfq_clean |> 
  filter(
    species_fish == "Lethrinus mahsena" & 
      size_cm > 15 & 
      landing_site == "Pwani"
    )

• The slice function allows to extract specific rows from a data frame.

• It offers flexibility for selecting rows by position or logical conditions.

• Select a row

lfq_clean |> 
  slice(1)

• Select range of rows

lfq_clean |> 
  slice(40:50)

• Select specific rows

lfq_clean |> 
  slice(c(5,25,400,605))

• negate (-) Used to exclude specific row

lfq_clean |> 
  slice(-1)

• negagte (-) to exclude specific rows

lfq_clean |> 
  slice(-c(5,25,400,605))

• select() function used for selecting specific variables from a data frame.
• It allows to create new data frames focused on relevant variables.
• Include column names directly within select().
• Use numeric vectors to select columns by position (base R style).
• Combine positive and negative values to include/exclude specific columns.

• Select columns

lfq_clean |> 
  select(species_fish, size_cm)

• select and rename

lfq_clean |> 
  select(
    site = landing_site,
    species = species_fish, 
    size = size_cm
    )

• negate (-) to deselect

lfq_clean |> 
  select(-species_fish, -size_cm)

• The mutate() function in {dplyr} adds new columns.
• It’s not destructive—all our existing data will still be there after you add new columns

• By default, mutate() sticks the new column on the far right of the dataset
• We can also control where the new column shows up with either the .before or .after argument:

something

something

• The group_by() function splits a dataset into smaller subsets based on the variable specified.

• This splitting happens behind the scenes—you don’t actually ever see the data split up into smaller datasets.
• To undo the grouping and bring all the rows back together, use ungroup().

• If we use mutate() after grouping, new columns are added to each subset separately.

• In many cases, you won’t notice any difference between using mutate() on an ungrouped or grouped dataset—you’ll get the same values.

• The summarize() function, is destructive.
• It collapses our dataset into a single value and throws away any columns that we don’t use when summarizing.

• All other variables are dropped and only the computed ones remains

• summarize() is better used with groups.
• If we use summarize() on a grouped dataset, each subset is collapsed into a single row.
• This will create different summary values, depending on the groups you use.
• For instance we want to know the number and average size of species landed

• We can specify more than one group with group_by()
• It create summary for each unique combination of values in the groups.
• For instance, if we want compute the number of observations and average size of fish by both specied of fish and site

• A mutating join allows you to combine variables from two tables.
• It first matches observations by their keys, then copies across variables from one table to the other.

• All rows from x where there are matching values in y, and all columns from x and y.

inner_join(x, y, by = "id")

• All rows from x, and all columns from x and y. Rows in x with no match in y will have NA values in the new columns.

left_join(x, y, by = "id")

• All rows from y, and all columns from x and y.
• Rows in y with no match in x will have NA values in the new columns.

right_join(x, y, by = "id")

• All rows and all columns from both x and y.
• Where there are not matching values, returns NA for the one missing.

full_join(x, y, by = "id")

• All rows from x where there are matching values in y, keeping just columns from x.

semi_join(x, y, by = "id")

• All rows from x where there are not matching values in y, keeping just columns from x.

anti_join(x, y, by = "id")

• Set operations are occasionally useful when you want to break a single complex filter into simpler pieces.
• All these operations work with a complete row, comparing the values of every variable.
• These expect the x and y inputs to have the same variables, and treat the observations like sets.

All rows from x and y, keeping duplicates.

• Common rows in both x and y, keeping just unique rows.

• All rows from x which are not also rows in y, keeping just unique rows.

• pivot_wider() is the opposite of pivot_longer()
• it makes a dataset wider by increasing the number of columns and decreasing the number of rows.
• t’s relatively rare to need pivot_wider() to make tidy data

• pivot_longer() makes datasets longer by increasing the number of rows and decreasing the number of columns.