Lecture 02

.title[
# Lecture 02
]
.subtitle[
## Shell and Julia
]
.author[
### Ivan Rudik
]
.date[
### AEM 7130
]

---

```r
if (!require("pacman")) install.packages("pacman")
```

```
## Loading required package: pacman
```

```r
pacman::p_load(
  xaringanExtra, JuliaCall
)
options(htmltools.dir.version = FALSE)
library(knitr)
opts_chunk$set(
  prompt = T, ## See hook below. I basically want a "$" prompt for every bash command in this lecture.
  fig.align = "center", fig.width=10, fig.height=6, 
  out.width="748px", out.length="520.75px",
  dpi = 300, #fig.path='Figs/',
  cache = F#, echo=F, warning=F, message=F
  )
## Next hook based on this SO answer: https://stackoverflow.com/a/39025054
knit_hooks$set(
  prompt = function(before, options, envir) {
    options(
      prompt = if (options$engine %in% c('sh','bash')) '$ ' else ' ',
      continue = if (options$engine %in% c('sh','bash')) '$ ' else ' '
      )
})
julia_setup(JULIA_HOME = "/Users/ir229/.juliaup/bin")
```

```
## Julia version 1.8.5 at location /Users/ir229/.julia/juliaup/julia-1.8.5+0.aarch64.apple.darwin14/bin will be used.
```

```
## Loading setup script for JuliaCall...
```

```
## Finish loading setup script for JuliaCall.
```

---

# Software and stuff

Necessary things to do:

- Install [VSCode](http://code.visualstudio.com) with these extensions: `Julia, Git Graph, Project Manager`

- Windows users: Install [Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10) and a Unix distribution or use https://repl.it

- Install [Julia](https://julialang.org/downloads/) with these Julia packages: `Expectations, Distributions, LinearAlgebra, BenchmarkTools`
    - Or use homebrew/other package manager

---

# Visual Studio Code

---

# VSCode

- Extensions
- Terminal/command line
- Julia (interactive)
- Project manager
- Git

---

# The shell

---

# Why learn the shell?

What is the shell?

The shell is the interface for interacting with your operating system,
typically we are referring to the command line interface (terminal, command prompt, bash, etc)

A lot of what you can do in the shell can be done in Julia itself, why bother with it?

---

# Why learn the shell?

Not everything can be done directily in your usual programming language

Command line is fast, powerful, and relatively easy to use,
especially with modern shells like zsh or fish

Writing shell scripts is reproducible and fast, unlike clicking buttons on a GUI

If you want to use servers or any high performance computing you are likely to need to use shell

---

# Why learn the shell?

You can automate your entire research pipeline with shell scripts (e.g. write something that calls multiple languages to execute your code then compiles your latex for the paper)

It really gets to the fundamentals of interacting with a computer (loops, tab-completions, saving scripts, etc)

It gets you understanding how to write code in terms of functions which will be important for any programming you do in scripting languages like Julia, R, MATLAB, or Stata

---

# What is the shell?

The shell is basically just a program where you can
type in commands to interact with the **kernel** and hardware

---

# What is the shell?

The most common one is .hi[bash], Bourne again shell, because it comes default on Linux and old Macs

I use .hi[fish], friendly interactive shell, because it comes default with a lot of nice features,
all the commands still work identically to bash

---

# Shell basics

When you open up the shell you should see a prompt, usually starting with `$ ` (don't type this)

```bash
$ 
```

---

# Shell basics

We can type in one .hi[command], `ls` which lists the contents of your current directory

```bash
$ ls
```

```
## 02-coding.Rmd
## 02-coding.html
## 02-coding_files
## figures
## my-css.css
## sandbox
```

My current directory is the one for this set of slides

---

# Shell basics

Commands come with potential .hi[options] or .hi[flags] that modify how they act

```bash
$ ls
```

```
## 02-coding.Rmd
## 02-coding.html
## 02-coding_files
## figures
## my-css.css
## sandbox
```

```bash
$ ls -l # long form command
```

```
## total 368
## -rw-r--r--   1 ir229  CORNELL\Domain Users   44780 Jan 31 15:42 02-coding.Rmd
## -rw-r--r--@  1 ir229  CORNELL\Domain Users  125586 Jan 31 15:44 02-coding.html
## drwxr-xr-x   4 ir229  CORNELL\Domain Users     128 Jan 31 15:44 02-coding_files
## drwxr-xr-x   8 ir229  CORNELL\Domain Users     256 Jul 12  2022 figures
## -rw-r--r--@  1 ir229  CORNELL\Domain Users    8387 Jan 22 17:44 my-css.css
## drwxr-xr-x  10 ir229  CORNELL\Domain Users     320 Jan 31 15:40 sandbox
```

---

# Shell basics

Options start with a dash and then a sequence of letters denoting which options you want

e.g. this lists files in long form `l`, sorted descending by size (`S`), with sizes in a human-readable format (`h`)

```bash
$ ls -lSh
```

```
## total 368
## -rw-r--r--@  1 ir229  CORNELL\Domain Users   123K Jan 31 15:44 02-coding.html
## -rw-r--r--   1 ir229  CORNELL\Domain Users    44K Jan 31 15:42 02-coding.Rmd
## -rw-r--r--@  1 ir229  CORNELL\Domain Users   8.2K Jan 22 17:44 my-css.css
## drwxr-xr-x  10 ir229  CORNELL\Domain Users   320B Jan 31 15:40 sandbox
## drwxr-xr-x   8 ir229  CORNELL\Domain Users   256B Jul 12  2022 figures
## drwxr-xr-x   4 ir229  CORNELL\Domain Users   128B Jan 31 15:44 02-coding_files
```

---

# Shell basics

Finally commands have an .hi[argument] that the command operates on

The previous `ls` calls were operating on the current directory,  
but we could use it on any directory we want

```bash
$ ls -lSh ~/Desktop/git
```

```
## total 0
## drwxr-xr-x  73 ir229  CORNELL\Domain Users   2.3K Jan 29 09:05 climate-trade
## drwxr-xr-x  26 ir229  CORNELL\Domain Users   832B Mar 22  2022 aem2850
## drwxr-xr-x@ 26 ir229  CORNELL\Domain Users   832B Jan 23 10:54 aem4510
## drwxr-xr-x  20 ir229  CORNELL\Domain Users   640B Nov 30 15:21 irudik.github.io
## drwxr-xr-x  19 ir229  CORNELL\Domain Users   608B Jan 29 14:18 hurricane-forecasts
## drwxr-xr-x  19 ir229  CORNELL\Domain Users   608B Nov 15 20:52 lead-education
## drwxr-xr-x  18 ir229  CORNELL\Domain Users   576B Dec 28 13:31 enviro-transport
## drwxr-xr-x@ 16 ir229  CORNELL\Domain Users   512B Jan 31 15:46 aem7130
## drwxr-xr-x  14 ir229  CORNELL\Domain Users   448B May 25  2022 biodiversity
## drwxr-xr-x  14 ir229  CORNELL\Domain Users   448B Jan 25 20:06 climate-networks
## drwxr-xr-x  14 ir229  CORNELL\Domain Users   448B Oct 24 17:26 regulation-markups
## drwxr-xr-x  12 ir229  CORNELL\Domain Users   384B Jul 12  2022 aem6510
## drwxr-xr-x  12 ir229  CORNELL\Domain Users   384B Aug 25 10:11 scc-structural
## drwxr-xr-x  10 ir229  CORNELL\Domain Users   320B Oct 27 21:43 0.ARCHIVE
## drwxr-xr-x   9 ir229  CORNELL\Domain Users   288B Aug 14 17:59 amenities-complementarity
## drwxr-xr-x   9 ir229  CORNELL\Domain Users   288B May 13  2022 climate-concentration
## drwxr-xr-x   9 ir229  CORNELL\Domain Users   288B May 20  2022 dropbox-template
## drwxr-xr-x   9 ir229  CORNELL\Domain Users   288B May 20  2022 repo-template
## drwxr-xr-x   7 ir229  CORNELL\Domain Users   224B Sep  4 15:28 climate-reading-group
## drwxr-xr-x   7 ir229  CORNELL\Domain Users   224B Feb 14  2022 config-files
```

---

# Shell basics

To see what commands and their options do, use the `man` (manual) command

`q` exits the man page, and spacebar lets you skip down by a page

```bash
$ man ls
```

```
## LS(1)                        General Commands Manual                       LS(1)
## 
## NNAAMMEE
##      llss – list directory contents
## 
## SSYYNNOOPPSSIISS
##      llss [--@@AABBCCFFGGHHIILLOOPPRRSSTTUUWWaabbccddeeffgghhiikkllmmnnooppqqrrssttuuvvwwxxyy11%%,,] [----ccoolloorr=_w_h_e_n]
##         [--DD _f_o_r_m_a_t] [_f_i_l_e _._._.]
## 
## DDEESSCCRRIIPPTTIIOONN
##      For each operand that names a _f_i_l_e of a type other than directory, llss
##      displays its name as well as any requested, associated information.  For
##      each operand that names a _f_i_l_e of type directory, llss displays the names of
##      files contained within that directory, as well as any requested, associated
##      information.
## 
##      If no operands are given, the contents of the current directory are
##      displayed.  If more than one operand is given, non-directory operands are
##      displayed first; directory and non-directory operands are sorted separately
##      and in lexicographical order.
## 
##      The following options are available:
## 
##      --@@      Display extended attribute keys and sizes in long (--ll) output.
## 
##      --AA      Include directory entries whose names begin with a dot (‘_.’) except
##              for _. and _._..  Automatically set for the super-user unless --II is
##              specified.
## 
##      --BB      Force printing of non-printable characters (as defined by ctype(3)
##              and current locale settings) in file names as \_x_x_x, where _x_x_x is
##              the numeric value of the character in octal.  This option is not
##              defined in IEEE Std 1003.1-2008 (“POSIX.1”).
## 
##      --CC      Force multi-column output; this is the default when output is to a
##              terminal.
## 
##      --DD _f_o_r_m_a_t
##              When printing in the long (--ll) format, use _f_o_r_m_a_t to format the
##              date and time output.  The argument _f_o_r_m_a_t is a string used by
##              strftime(3).  Depending on the choice of format string, this may
##              result in a different number of columns in the output.  This option
##              overrides the --TT option.  This option is not defined in IEEE Std
##              1003.1-2008 (“POSIX.1”).
## 
##      --FF      Display a slash (‘/’) immediately after each pathname that is a
##              directory, an asterisk (‘*’) after each that is executable, an at
##              sign (‘@’) after each symbolic link, an equals sign (‘=’) after
##              each socket, a percent sign (‘%’) after each whiteout, and a
##              vertical bar (‘|’) after each that is a FIFO.
## 
##      --GG      Enable colorized output.  This option is equivalent to defining
##              CLICOLOR or COLORTERM in the environment and setting ----ccoolloorr=_a_u_t_o.
##              (See below.)  This functionality can be compiled out by removing
##              the definition of COLORLS.  This option is not defined in IEEE Std
##              1003.1-2008 (“POSIX.1”).
## 
##      --HH      Symbolic links on the command line are followed.  This option is
##              assumed if none of the --FF, --dd, or --ll options are specified.
## 
##      --II      Prevent --AA from being automatically set for the super-user.  This
##              option is not defined in IEEE Std 1003.1-2008 (“POSIX.1”).
## 
##      --LL      Follow all symbolic links to final target and list the file or
##              directory the link references rather than the link itself.  This
##              option cancels the --PP option.
## 
##      --OO      Include the file flags in a long (--ll) output.  This option is
##              incompatible with IEEE Std 1003.1-2008 (“POSIX.1”).  See chflags(1)
##              for a list of file flags and their meanings.
## 
##      --PP      If argument is a symbolic link, list the link itself rather than
##              the object the link references.  This option cancels the --HH and --LL
##              options.
## 
##      --RR      Recursively list subdirectories encountered.
## 
##      --SS      Sort by size (largest file first) before sorting the operands in
##              lexicographical order.
## 
##      --TT      When printing in the long (--ll) format, display complete time
##              information for the file, including month, day, hour, minute,
##              second, and year.  The --DD option gives even more control over the
##              output format.  This option is not defined in IEEE Std 1003.1-2008
##              (“POSIX.1”).
## 
##      --UU      Use time when file was created for sorting or printing.  This
##              option is not defined in IEEE Std 1003.1-2008 (“POSIX.1”).
## 
##      --WW      Display whiteouts when scanning directories.  This option is not
##              defined in IEEE Std 1003.1-2008 (“POSIX.1”).
## 
##      --aa      Include directory entries whose names begin with a dot (‘_.’).
## 
##      --bb      As --BB, but use C escape codes whenever possible.  This option is
##              not defined in IEEE Std 1003.1-2008 (“POSIX.1”).
## 
##      --cc      Use time when file status was last changed for sorting or printing.
## 
##      ----ccoolloorr=_w_h_e_n
##              Output colored escape sequences based on _w_h_e_n, which may be set to
##              either aallwwaayyss, aauuttoo, or nneevveerr.
## 
##              aallwwaayyss will make llss always output color.  If TERM is unset or set
##              to an invalid terminal, then llss will fall back to explicit ANSI
##              escape sequences without the help of termcap(5).  aallwwaayyss is the
##              default if ----ccoolloorr is specified without an argument.
## 
##              aauuttoo will make llss output escape sequences based on termcap(5), but
##              only if stdout is a tty and either the --GG flag is specified or the
##              COLORTERM environment variable is set and not empty.
## 
##              nneevveerr will disable color regardless of environment variables.
##              nneevveerr is the default when neither ----ccoolloorr nor --GG is specified.
## 
##              For compatibility with GNU coreutils, llss supports yyeess or ffoorrccee as
##              equivalent to aallwwaayyss, nnoo or nnoonnee as equivalent to nneevveerr, and ttttyy or
##              iiff--ttttyy as equivalent to aauuttoo.
## 
##      --dd      Directories are listed as plain files (not searched recursively).
## 
##      --ee      Print the Access Control List (ACL) associated with the file, if
##              present, in long (--ll) output.
## 
##      --ff      Output is not sorted.  This option turns on --aa.  It also negates
##              the effect of the --rr, --SS and --tt options.  As allowed by IEEE Std
##              1003.1-2008 (“POSIX.1”), this option has no effect on the --dd, --ll,
##              --RR and --ss options.
## 
##      --gg      This option has no effect.  It is only available for compatibility
##              with 4.3BSD, where it was used to display the group name in the
##              long (--ll) format output.  This option is incompatible with IEEE Std
##              1003.1-2008 (“POSIX.1”).
## 
##      --hh      When used with the --ll option, use unit suffixes: Byte, Kilobyte,
##              Megabyte, Gigabyte, Terabyte and Petabyte in order to reduce the
##              number of digits to four or fewer using base 2 for sizes.  This
##              option is not defined in IEEE Std 1003.1-2008 (“POSIX.1”).
## 
##      --ii      For each file, print the file's file serial number (inode number).
## 
##      --kk      This has the same effect as setting environment variable BLOCKSIZE
##              to 1024, except that it also nullifies any --hh options to its left.
## 
##      --ll      (The lowercase letter “ell”.) List files in the long format, as
##              described in the _T_h_e _L_o_n_g _F_o_r_m_a_t subsection below.
## 
##      --mm      Stream output format; list files across the page, separated by
##              commas.
## 
##      --nn      Display user and group IDs numerically rather than converting to a
##              user or group name in a long (--ll) output.  This option turns on the
##              --ll option.
## 
##      --oo      List in long format, but omit the group id.
## 
##      --pp      Write a slash (‘/’) after each filename if that file is a
##              directory.
## 
##      --qq      Force printing of non-graphic characters in file names as the
##              character ‘?’; this is the default when output is to a terminal.
## 
##      --rr      Reverse the order of the sort.
## 
##      --ss      Display the number of blocks used in the file system by each file.
##              Block sizes and directory totals are handled as described in _T_h_e
##              _L_o_n_g _F_o_r_m_a_t subsection below, except (if the long format is not
##              also requested) the directory totals are not output when the output
##              is in a single column, even if multi-column output is requested.
##              (--ll) format, display complete time information for the file,
##              including month, day, hour, minute, second, and year.  The --DD
##              option gives even more control over the output format.  This option
##              is not defined in IEEE Std 1003.1-2008 (“POSIX.1”).
## 
##      --tt      Sort by descending time modified (most recently modified first).
##              If two files have the same modification timestamp, sort their names
##              in ascending lexicographical order.  The --rr option reverses both of
##              these sort orders.
## 
##              Note that these sort orders are contradictory: the time sequence is
##              in descending order, the lexicographical sort is in ascending
##              order.  This behavior is mandated by IEEE Std 1003.2 (“POSIX.2”).
##              This feature can cause problems listing files stored with
##              sequential names on FAT file systems, such as from digital cameras,
##              where it is possible to have more than one image with the same
##              timestamp.  In such a case, the photos cannot be listed in the
##              sequence in which they were taken.  To ensure the same sort order
##              for time and for lexicographical sorting, set the environment
##              variable LS_SAMESORT or use the --yy option.  This causes llss to
##              reverse the lexicographical sort order when sorting files with the
##              same modification timestamp.
## 
##      --uu      Use time of last access, instead of time of last modification of
##              the file for sorting (--tt) or long printing (--ll).
## 
##      --vv      Force unedited printing of non-graphic characters; this is the
##              default when output is not to a terminal.
## 
##      --ww      Force raw printing of non-printable characters.  This is the
##              default when output is not to a terminal.  This option is not
##              defined in IEEE Std 1003.1-2001 (“POSIX.1”).
## 
##      --xx      The same as --CC, except that the multi-column output is produced
##              with entries sorted across, rather than down, the columns.
## 
##      --yy      When the --tt option is set, sort the alphabetical output in the same
##              order as the time output.  This has the same effect as setting
##              LS_SAMESORT.  See the description of the --tt option for more
##              details.  This option is not defined in IEEE Std 1003.1-2001
##              (“POSIX.1”).
## 
##      --%%      Distinguish dataless files and directories with a '%' character in
##              long
## 
##      --11      (The numeric digit “one”.) Force output to be one entry per line.
##              This is the default when output is not to a terminal.  (--ll) output,
##              and don't materialize dataless directories when listing them.
## 
##      --,      (Comma) When the --ll option is set, print file sizes grouped and
##              separated by thousands using the non-monetary separator returned by
##              localeconv(3), typically a comma or period.  If no locale is set,
##              or the locale does not have a non-monetary separator, this option
##              has no effect.  This option is not defined in IEEE Std 1003.1-2001
##              (“POSIX.1”).
## 
##      The --11, --CC, --xx, and --ll options all override each other; the last one
##      specified determines the format used.
## 
##      The --cc, --uu, and --UU options all override each other; the last one specified
##      determines the file time used.
## 
##      The --SS and --tt options override each other; the last one specified
##      determines the sort order used.
## 
##      The --BB, --bb, --ww, and --qq options all override each other; the last one
##      specified determines the format used for non-printable characters.
## 
##      The --HH, --LL and --PP options all override each other (either partially or
##      fully); they are applied in the order specified.
## 
##      By default, llss lists one entry per line to standard output; the exceptions
##      are to terminals or when the --CC or --xx options are specified.
## 
##      File information is displayed with one or more ⟨blank⟩s separating the
##      information associated with the --ii, --ss, and --ll options.
## 
##    TThhee LLoonngg FFoorrmmaatt
##      If the --ll option is given, the following information is displayed for each
##      file: file mode, number of links, owner name, group name, number of bytes
##      in the file, abbreviated month, day-of-month file was last modified, hour
##      file last modified, minute file last modified, and the pathname.  If the
##      file or directory has extended attributes, the permissions field printed by
##      the --ll option is followed by a '@' character.  Otherwise, if the file or
##      directory has extended security information (such as an access control
##      list), the permissions field printed by the --ll option is followed by a '+'
##      character.  If the --%% option is given, a '%' character follows the
##      permissions field for dataless files and directories, possibly replacing
##      the '@' or '+' character.
## 
##      If the modification time of the file is more than 6 months in the past or
##      future, and the --DD or --TT are not specified, then the year of the last
##      modification is displayed in place of the hour and minute fields.
## 
##      If the owner or group names are not a known user or group name, or the --nn
##      option is given, the numeric ID's are displayed.
## 
##      If the file is a character special or block special file, the device number
##      for the file is displayed in the size field.  If the file is a symbolic
##      link the pathname of the linked-to file is preceded by “->”.
## 
##      The listing of a directory's contents is preceded by a labeled total number
##      of blocks used in the file system by the files which are listed as the
##      directory's contents (which may or may not include _. and _._. and other files
##      which start with a dot, depending on other options).
## 
##      The default block size is 512 bytes.  The block size may be set with option
##      --kk or environment variable BLOCKSIZE.  Numbers of blocks in the output will
##      have been rounded up so the numbers of bytes is at least as many as used by
##      the corresponding file system blocks (which might have a different size).
## 
##      The file mode printed under the --ll option consists of the entry type and
##      the permissions.  The entry type character describes the type of file, as
##      follows:
## 
##            --     Regular file.
##            bb     Block special file.
##            cc     Character special file.
##            dd     Directory.
##            ll     Symbolic link.
##            pp     FIFO.
##            ss     Socket.
##            ww     Whiteout.
## 
##      The next three fields are three characters each: owner permissions, group
##      permissions, and other permissions.  Each field has three character
##      positions:
## 
##            1.   If rr, the file is readable; if --, it is not readable.
## 
##            2.   If ww, the file is writable; if --, it is not writable.
## 
##            3.   The first of the following that applies:
## 
##                       SS     If in the owner permissions, the file is not
##                             executable and set-user-ID mode is set.  If in the
##                             group permissions, the file is not executable and
##                             set-group-ID mode is set.
## 
##                       ss     If in the owner permissions, the file is executable
##                             and set-user-ID mode is set.  If in the group
##                             permissions, the file is executable and setgroup-ID
##                             mode is set.
## 
##                       xx     The file is executable or the directory is
##                             searchable.
## 
##                       --     The file is neither readable, writable, executable,
##                             nor set-user-ID nor set-group-ID mode, nor sticky.
##                             (See below.)
## 
##                 These next two apply only to the third character in the last
##                 group (other permissions).
## 
##                       TT     The sticky bit is set (mode 1000), but not execute
##                             or search permission.  (See chmod(1) or sticky(7).)
## 
##                       tt     The sticky bit is set (mode 1000), and is searchable
##                             or executable.  (See chmod(1) or sticky(7).)
## 
##      The next field contains a plus (‘+’) character if the file has an ACL, or a
##      space (‘ ’) if it does not.  The llss utility does not show the actual ACL;
##      use getfacl(1) to do this.
## 
## EENNVVIIRROONNMMEENNTT
##      The following environment variables affect the execution of llss:
## 
##      BLOCKSIZE           If this is set, its value, rounded up to 512 or down to
##                          a multiple of 512, will be used as the block size in
##                          bytes by the --ll and --ss options.  See _T_h_e _L_o_n_g _F_o_r_m_a_t
##                          subsection for more information.
## 
##      CLICOLOR            Use ANSI color sequences to distinguish file types.
##                          See LSCOLORS below.  In addition to the file types
##                          mentioned in the --FF option some extra attributes
##                          (setuid bit set, etc.) are also displayed.  The
##                          colorization is dependent on a terminal type with the
##                          proper termcap(5) capabilities.  The default “cons25”
##                          console has the proper capabilities, but to display the
##                          colors in an xterm(1), for example, the TERM variable
##                          must be set to “xterm-color”.  Other terminal types may
##                          require similar adjustments.  Colorization is silently
##                          disabled if the output is not directed to a terminal
##                          unless the CLICOLOR_FORCE variable is defined or
##                          ----ccoolloorr is set to “always”.
## 
##      CLICOLOR_FORCE      Color sequences are normally disabled if the output is
##                          not directed to a terminal.  This can be overridden by
##                          setting this variable.  The TERM variable still needs
##                          to reference a color capable terminal however otherwise
##                          it is not possible to determine which color sequences
##                          to use.
## 
##      COLORTERM           See description for CLICOLOR above.
## 
##      COLUMNS             If this variable contains a string representing a
##                          decimal integer, it is used as the column position
##                          width for displaying multiple-text-column output.  The
##                          llss utility calculates how many pathname text columns to
##                          display based on the width provided.  (See --CC and --xx.)
## 
##      LANG                The locale to use when determining the order of day and
##                          month in the long --ll format output.  See environ(7) for
##                          more information.
## 
##      LSCOLORS            The value of this variable describes what color to use
##                          for which attribute when colors are enabled with
##                          CLICOLOR or COLORTERM.  This string is a concatenation
##                          of pairs of the format _f_b, where _f is the foreground
##                          color and _b is the background color.
## 
##                          The color designators are as follows:
## 
##                                aa     black
##                                bb     red
##                                cc     green
##                                dd     brown
##                                ee     blue
##                                ff     magenta
##                                gg     cyan
##                                hh     light grey
##                                AA     bold black, usually shows up as dark grey
##                                BB     bold red
##                                CC     bold green
##                                DD     bold brown, usually shows up as yellow
##                                EE     bold blue
##                                FF     bold magenta
##                                GG     bold cyan
##                                HH     bold light grey; looks like bright white
##                                xx     default foreground or background
## 
##                          Note that the above are standard ANSI colors.  The
##                          actual display may differ depending on the color
##                          capabilities of the terminal in use.
## 
##                          The order of the attributes are as follows:
## 
##                                1.   directory
##                                2.   symbolic link
##                                3.   socket
##                                4.   pipe
##                                5.   executable
##                                6.   block special
##                                7.   character special
##                                8.   executable with setuid bit set
##                                9.   executable with setgid bit set
##                                10.  directory writable to others, with sticky
##                                     bit
##                                11.  directory writable to others, without sticky
##                                     bit
## 
##                          The default is "exfxcxdxbxegedabagacad", i.e., blue
##                          foreground and default background for regular
##                          directories, black foreground and red background for
##                          setuid executables, etc.
## 
##      LS_COLWIDTHS        If this variable is set, it is considered to be a
##                          colon-delimited list of minimum column widths.
##                          Unreasonable and insufficient widths are ignored (thus
##                          zero signifies a dynamically sized column).  Not all
##                          columns have changeable widths.  The fields are, in
##                          order: inode, block count, number of links, user name,
##                          group name, flags, file size, file name.
## 
##      LS_SAMESORT         If this variable is set, the --tt option sorts the names
##                          of files with the same modification timestamp in the
##                          same sense as the time sort.  See the description of
##                          the --tt option for more details.
## 
##      TERM                The CLICOLOR and COLORTERM functionality depends on a
##                          terminal type with color capabilities.
## 
##      TZ                  The timezone to use when displaying dates.  See
##                          environ(7) for more information.
## 
## EEXXIITT SSTTAATTUUSS
##      The llss utility exits 0 on success, and >0 if an error occurs.
## 
## EEXXAAMMPPLLEESS
##      List the contents of the current working directory in long format:
## 
##            $ ls -l
## 
##      In addition to listing the contents of the current working directory in
##      long format, show inode numbers, file flags (see chflags(1)), and suffix
##      each filename with a symbol representing its file type:
## 
##            $ ls -lioF
## 
##      List the files in _/_v_a_r_/_l_o_g, sorting the output such that the most recently
##      modified entries are printed first:
## 
##            $ ls -lt /var/log
## 
## CCOOMMPPAATTIIBBIILLIITTYY
##      The group field is now automatically included in the long listing for files
##      in order to be compatible with the IEEE Std 1003.2 (“POSIX.2”)
##      specification.
## 
## LLEEGGAACCYY DDEESSCCRRIIPPTTIIOONN
##      In legacy mode, the --ff option does not turn on the --aa option and the --gg,
##      --nn, and --oo options do not turn on the --ll option.
## 
##      Also, the --oo option causes the file flags to be included in a long (-l)
##      output; there is no --OO option.
## 
##      When --HH is specified (and not overridden by --LL or --PP) and a file argument
##      is a symlink that resolves to a non-directory file, the output will reflect
##      the nature of the link, rather than that of the file.  In legacy operation,
##      the output will describe the file.
## 
##      For more information about legacy mode, see compat(5).
## 
## SSEEEE AALLSSOO
##      chflags(1), chmod(1), getfacl(1), sort(1), xterm(1), localeconv(3),
##      strftime(3), strmode(3), compat(5), termcap(5), sticky(7), symlink(7)
## 
## SSTTAANNDDAARRDDSS
##      With the exception of options --gg, --nn and --oo, the llss utility conforms to
##      IEEE Std 1003.1-2001 (“POSIX.1”) and IEEE Std 1003.1-2008 (“POSIX.1”).  The
##      options --BB, --DD, --GG, --II, --TT, --UU, --WW, --ZZ, --bb, --hh, --ww, --yy and --, are non-
##      standard extensions.
## 
##      The ACL support is compatible with IEEE Std 1003.2c (“POSIX.2c”) Draft 17
##      (withdrawn).
## 
## HHIISSTTOORRYY
##      An llss command appeared in Version 1 AT&T UNIX.
## 
## BBUUGGSS
##      To maintain backward compatibility, the relationships between the many
##      options are quite complex.
## 
##      The exception mentioned in the --ss option description might be a feature
##      that was based on the fact that single-column output usually goes to
##      something other than a terminal.  It is debatable whether this is a design
##      bug.
## 
##      IEEE Std 1003.2 (“POSIX.2”) mandates opposite sort orders for files with
##      the same timestamp when sorting with the --tt option.
## 
## macOS 12.6                       August 31, 2020                      macOS 12.6
```

---

# Shell basics

Pressing `h` within a man page brings up the help page for how to navigate them

`/terms_here` lets you search within the man page for particular terms

Use `n` and `shift+n` to move forward and backward between matches

---

# Navigation

We already learned how to list the files in a particular directory, but we need a few other tools to navigate around our machine

We often will want to know our .hi[current working directory] so we know where we are before we start running commands

We do this with `pwd`

```bash
$ pwd
```

```
## /Users/ir229/Desktop/git/aem7130/lecture-notes/02-coding
```

---

# Navigation

Directories are organized in a hierarchical structure, at the top is the root directory, `/`

```bash
$ ls /
```

```
## Applications
## Library
## Network
## System
## Users
## Volumes
## bin
## cores
## dev
## etc
## home
## opt
## private
## sbin
## tmp
## usr
## var
```

---

# Navigation

The root directory contains everything else

Other directories are inside the root directory and come afterward in the file path separated by forward slashes `/`

```bash
$ ls -lSh /Users
```

```
## total 0
## drwxr-x---+ 68 ir229           CORNELL\Domain Users   2.1K Jan 29 09:05 ir229
## drwxr-x---+ 19 cals_oit        staff                  608B Feb  9  2022 cals_oit
## drwxr-x---+ 15 ag-jjp34-doc10  CORNELL\Domain Users   480B Feb  9  2022 ag-jjp34-doc10
## drwxrwxrwt  14 root            wheel                  448B Jan 17 17:21 Shared
```

---

# Navigation

The root directory contains everything else

Other directories are inside the root directory and come afterward in the file path separated by forward slashes `/`

```bash
$ ls -lSh /Users/ir229
```

```
## total 0
## drwx------@ 453 ir229  CORNELL\Domain Users    14K Jan 31 15:30 Downloads
## drwx------@  93 ir229  CORNELL\Domain Users   2.9K May 20  2022 Library
## drwx------@  43 ir229  CORNELL\Domain Users   1.3K Jan 18 14:47 Dropbox
## drwxr-xr-x   33 ir229  CORNELL\Domain Users   1.0K Dec 12 14:52 languageserver-library
## drwxr-xr-x   18 ir229  CORNELL\Domain Users   576B Jan 17 17:21 Zotero
## drwx------@  16 ir229  CORNELL\Domain Users   512B Sep  8 12:39 Box-Box-Backup
## drwx------@   6 ir229  CORNELL\Domain Users   192B Jan 10 08:11 Desktop
## drwx------+   5 ir229  CORNELL\Domain Users   160B Oct  5 12:03 Documents
## drwx------    5 ir229  CORNELL\Domain Users   160B May 10  2022 Movies
## drwx------+   5 ir229  CORNELL\Domain Users   160B May 10  2022 Music
## drwx------+   5 ir229  CORNELL\Domain Users   160B May 10  2022 Pictures
## drwxr-xr-x    4 ir229  CORNELL\Domain Users   128B May 10  2022 Applications
## drwx------@   4 ir229  CORNELL\Domain Users   128B Nov  5 13:42 Creative Cloud Files
## drwx------@   4 ir229  CORNELL\Domain Users   128B Aug  2  2022 Creative Cloud Files irudik@gmail.com ffd387f23bd1f53514a9de8d75abe8c7fb5ffaa6109df102045adaf1928dcdd9
## drwxr-xr-x+   4 ir229  CORNELL\Domain Users   128B Feb 14  2022 Public
```

---

# Navigation

Next we need to be able to change directories, we can do this with `cd` (change directory)

```bash
$ cd /Users/ir229/Desktop/git
$ pwd
```

```
## /Users/ir229/Desktop/git
```

When navigating, it is often easier and more reproducible to use .hi[relative paths]

This is when arguments are relative to your current working directory, instead of using absolute paths (e.g. /Users/ir229/Desktop/git)

---

# Navigation

There's a few expressions that make this possible

- `~` is your .hi[home directory]
- `.` is your current directory
- `..` is the parent directory
- `-` is the previous directory you were in

---

# Navigation

```bash
$ cd ~ # move to home directory (will vary computer-to-computer)
$ pwd
$ cd - # move to previous directory (lecture notes 2 directory)
$ cd .. # move to parent directory (general lecture notes directory)
$ pwd
$ cd . # move to current directory (nothing changes)
$ pwd
```

```
## /Users/ir229
## /Users/ir229/Desktop/git/aem7130/lecture-notes/02-coding
## /Users/ir229/Desktop/git/aem7130/lecture-notes
## /Users/ir229/Desktop/git/aem7130/lecture-notes
```

You can see `.` and `..` in your current directory when using `ls` with the `a` flag

```bash
$ ls -a
```

```
## .
## ..
## .DS_Store
## 02-coding.Rmd
## 02-coding.html
## 02-coding_files
## figures
## my-css.css
## sandbox
```
---

# Navigation

This makes navigation much easier

If we wanted to move from the current directory to the parent directory for this year's course we can just do

```bash
$ pwd
$ cd ../..
$ pwd
```

```
## /Users/ir229/Desktop/git/aem7130/lecture-notes/02-coding
## /Users/ir229/Desktop/git/aem7130
```

instead of

```bash
$ cd /Users/ir229/Desktop/git/aem7130
$ pwd
```

```
## /Users/ir229/Desktop/git/aem7130
```

---

# Navigation

Relative paths are very important for reproducible code

Your directory structure starting at root `/` will not be the same as someone else's

Using relative paths lets you circumvent this as long as the project's directory structure is consistent

If you use Git or Dropbox it should be

---

# Creating files and directories

We learned how to move around directories but how do we make them?

We do so with `mkdir` (make directory)

```bash
$ mkdir test_directory
$ ls
```

```
## 02-coding.Rmd
## 02-coding.html
## 02-coding_files
## figures
## my-css.css
## sandbox
## test_directory
```

---

# Creating files and directories

We create blank files using `touch`

```bash
$ touch test_directory/test.txt test_directory/test1.txt
$ ls test_directory
```

```
## test.txt
## test1.txt
```

`touch` is useful if you have a program that can't create a file itself but can edit them

---

# Creating files and directories

If you have a Unix system pre-installed with nano
you can use `nano` to create and edit the file

```bash
$ nano test_directory/test.txt
```

---

# Creating files and directories

Here are some tips for naming files and directories

1. .hi-red[DON'T USE SPACES]
    - Spaces are used to separate commands, you generally want to avoid them in names in favor of underscores or dashes
2. Use letters, numbers, underscores, periods, and dashes only

---

# Creating files and directories

If you really, really, want to use spaces in names you'll have to do one of two things, enclose in quotes or backslash the space

```bash
$ mkdir "123test directory"
$ ls
```

```
## 02-coding.Rmd
## 02-coding.html
## 02-coding_files
## 123test directory
## figures
## my-css.css
## sandbox
## test_directory
```

---

# Creating files and directories

If you really, really, want to use spaces in names you'll have to do one of two things, enclose in quotes or backslash the space

```bash
$ mkdir 123test\ directory\ 2
$ ls
```

```
## 02-coding.Rmd
## 02-coding.html
## 02-coding_files
## 123test directory
## 123test directory 2
## figures
## my-css.css
## sandbox
## test_directory
```

```
## 02-coding.Rmd
## 02-coding.html
## 02-coding_files
## figures
## my-css.css
## sandbox
## test_directory
```

---