Show Lecture.BashScripts as a slide show.

CT320: Bash Scripts

Bash Scripts

• Shell Scripts
  • Variables and Quoting
  • Arguments and Functions
  • Control Flow
  • Arrays and Arithmetic
  • Regular Expressions

It’s a program

A shell script is a program. Therefore, it deserves all of the care that any other program should get, including, as appropriate:

• Meaningful variable names
• Indentation
• Modular functions
• Comments

Morons

Only an idiot would justify sloppy work with, “It’s only a shell script, so I didn’t bother doing …”.

Sure, if it’s a short-lived (you hope) program (whether script or not), then it may not require the full treatment. However, that decision is not determined by whether or not the program is a shell script, a Perl script, a Python script, or a C++ program.

Shell Scripting

Shell scripts are programs that:

• allow automation of complex tasks
• provide batch execution capability
• used to control periodic processes
• have no limit to their complexity
• have variables and control flow
• accept command line arguments
• read input and write output and files

Focus

There are many shells, including:

• Bourne Shell, sh
• Bourne-Again Shell, bash
• Korn Shell, ksh
• C Shell, csh
• Z Shell, zsh

From now on, we will focus on bash.

Input and Output

• echo "string" — send string to stdout
• read variable — read variable from stdin
  • Rarely used. Instead, use command-line arguments.

Variables and Quoting

• Variable names are unmarked in assignments
• Example: pathname="/usr/lib"
• Variable name is prefixed with $ to get its value

$ name=Bozo
$ let amount=2+7
$ echo "name amount"
name amount
$ echo "$name $amount"
Bozo 9
$ echo $UID
1038

Arguments

• Scripts accept command-line arguments, just like programs:
  • $0 — same as C’s argv[0]
  • $1 — same as C’s argv[1]
  • $2 — same as C’s argv[2]
  • …
  • $@ — same as "$1" "$2" "$3" …
  • $* — same as "$1 $2 $3 …"
  • $# — same as C’s argc-1

Script Syntax

• Initial line of file identifies which shell
• Example: #! /bin/bash
• Comments start with hash mark
• Example: # Make sure to remove all gophers here
• Functions
  • Allow grouping of commands
  • Can accept parameters
  • Can create local variables
  • Example: usage() { echo "Usage: $0 <param>" >&2; }

Arguments example

$ echo -e '#! /bin/bash\nlet sum=$1+$2\necho "Sum is $sum"' >s
$ cat s
#! /bin/bash
let sum=$1+$2
echo "Sum is $sum"
$ ./s 34 5 12
/tmp/pmwiki-bash-script7D1O7A: line 14: ./s: Permission denied
$ ls -l s
-rw------- 1 ct320 class 46 Nov 21 09:35 s
$ chmod +x s
$ ls -l s
-rwx------ 1 ct320 class 46 Nov 21 09:35 s
$ ./s 34 5 12
Sum is 39

if/then/else (numeric)

(( expression )) allows a C-like arithmetic expression as a condition:

cd /etc
let amount=$(ls | wc -l)
if (( amount == 0 ))
then
    echo "$PWD is empty."
elif ((amount==1))
then
    echo "$PWD contains one file."
else
    echo "$PWD contains $amount files."
fi

/etc contains 148 files.

case

Of course, as in most programming languages, there’s also a case construct:

cd /etc
let amount=$(ls | wc -l)
case $amount in
0) echo "$PWD is empty.";;
1) echo "$PWD contains one file.";;
*) echo "$PWD contains $amount files.";;
esac

/etc contains 148 files.

if/then/else (strings)

To compare strings, use [[ … ]]. Unlike (( … )), spaces are required.

if [[ $HOSTNAME = boston || $HOSTNAME = denver ]]
then
    echo "You are in the CSB 120 lab."
elif [[ $HOSTNAME = *[aeiouy]* ]]
then
    echo "Well, at least $HOSTNAME contains a vowel."
else
    echo "$HOSTNAME confuses me. ☹"
fi

Well, at least beethoven contains a vowel.

String comparison

• There are several string comparison operators:
  • [[ string = string ]]
  • [[ string = filename-pattern ]]
  • [[ string != string ]]
  • [[ string != filename-pattern ]]
  • [[ string < string ]]
  • [[ string > string ]]
  • [[ string =~ regular-expression ]]
• == may work the same as =, but is non-standard.
• A filename-pattern is not a regular-expression.
• Spaces around the operators!

if/then/else (files)

• Other things that can go inside [[ … ]]:
  • [[ -e path ]] — Does this exist?
  • [[ -f path ]] — Is this a plain file?
  • [[ -d path ]] — Is this a directory?
  • [[ -s path ]] — Does this have a non-zero size?
  • [[ -r path ]] — Can I read this?
  • [[ -w path ]] — Can I write this?
  • [[ -x path ]] — Can I execute this?

Filename expression example

Note that the operator is =. This does not mean to test for string equality, but, instead, to match the left side against the filename expression that is the right side.

if [[ $OSTYPE = linux* ]]
then
    echo "Great, we’re on something Linuxy: $OSTYPE"
else
    echo "Sorry, but the OS $OSTYPE is not supported."
fi

Great, we’re on something Linuxy: linux-gnu

Regular expression example

The =~ operator uses a regular expression, like the grep command. This is not a filename expression.

when=$(date)
if [[ $when =~ (Tue|Thu).*Aug ]]
then
    echo "Good."
else
    echo "Sorry, this only works on August Tuesdays/Thursdays."
    echo "Current date: $when"
fi

Sorry, this only works on August Tuesdays/Thursdays.
Current date: Thu Nov 21 09:35:01 MST 2024

$?

$ date
Thu Nov 21 09:35:01 MST 2024
$ echo $?
0
$ cp /etc/group xyz
$ echo $?
0
$ cp /etc/superman xyz
cp: cannot stat '/etc/superman': No such file or directory
$ echo $?
1
$ grep 'beet' /etc/hosts
129.82.44.240	dung-beetle.cs.colostate.edu dung-beetle
129.82.45.48	beethoven.cs.colostate.edu beethoven
$ echo $?
0
$ grep "cowabunga" /etc/hosts
$ echo $?
1
$ echo $?
0

• $? is the exit code of the most recent program.
• 0 means success.
• >0 means failure.
• The opposite of boolean.

if/then/else (program)

• In general, if/then/else uses a program.
• Often, this program is a built-in, such as [[ … ]] or (( … )).
• However, it doesn’t have to be.

if grep -q "colostate.edu" /etc/resolv.conf
then
    echo "operating at CSU"
else
    echo "must be at home or out & about"
fi

operating at CSU

if/then/else (program)

Here’s the same code, written by someone who doesn’t understand how if/then/else works:

grep -q "colostate.edu" /etc/resolv.conf
if (($? == 0))  # or, worse: [[ $? -eq 0 ]]
then
    echo "operating at CSU"
else
    echo "must be at home or out & about"
fi

operating at CSU

if/then/else (general)

You can combine all three syntaxes using &&, ||, and !:

if ((UID>0)) && [[ -r /etc/group ]] && ! cmp -s /bin/date /bin/sh
then
    echo "Holy Toledo, that was a miracle!"
fi

Holy Toledo, that was a miracle!

Loops use the same syntax

while loops use the same argument as if. That is, you can have:

• a string-based while loop
• a number-based while loop
• a program-based while loop

while loop example

let n=1
while ((n > 0))
do
    echo "n is $n"
    let n=n*32
done

n is 1
n is 32
n is 1024
n is 32768
n is 1048576
n is 33554432
n is 1073741824
n is 34359738368
n is 1099511627776
n is 35184372088832
n is 1125899906842624
n is 36028797018963968
n is 1152921504606846976

for loops (string-based)

for beatle in John Paul George Ringo
do
    echo "$beatle was one of the Beatles."
done

John was one of the Beatles.
Paul was one of the Beatles.
George was one of the Beatles.
Ringo was one of the Beatles.

for loops (string-based)

for datafile in /bin/c*s*u*
do
    echo Processing $datafile
done

Processing /bin/cimsub
Processing /bin/cinnamon-session-quit
Processing /bin/cinnamon-settings-users
Processing /bin/cinnamon-subprocess-wrapper
Processing /bin/cksum
Processing /bin/clang-pseudo
Processing /bin/clevis-decrypt-null
Processing /bin/clevis-encrypt-null
Processing /bin/clevis-luks-bind
Processing /bin/clevis-luks-common-functions
Processing /bin/clevis-luks-edit
Processing /bin/clevis-luks-list
Processing /bin/clevis-luks-pass
Processing /bin/clevis-luks-regen
Processing /bin/clevis-luks-report
Processing /bin/clevis-luks-unbind
Processing /bin/clevis-luks-unlock
Processing /bin/cmsutil
Processing /bin/cvsbug
Processing /bin/cvtsudoers

for loops (integer based)

A C-like arithmetic loop exists:

let max=5
for ((i=max; i>=0; i--))
do
    echo $i
done
echo "Blast off!"

5
4
3
2
1
0
Blast off!

Why not $max?

Arithmetic Loops

Or:

for i in {5..1}
do
    echo $i
done
echo "Blast off!"

5
4
3
2
1
Blast off!

Arithmetic evaluation

• Let’s say that we want to add two and five, and multiply the result by three.
• No, don’t do it with pencil & paper, or in your head.
• Don’t do work for the computer. The computer is your servant. Make it do the work.
• It’s probably more accurate that way.

Primary attempt

Let’s try some straightforward code:

$ alpha=2+5
$ beta=alpha*3
$ echo beta
beta

Secondary attempt

Of course, echo beta simply yields “beta”. Perhaps a $ will help to extract the value of the variable:

$ alpha=2+5
$ beta=alpha*3
$ echo $beta
alpha*3

Tertiary attempt

We got the string “alpha” instead of its value. Let’s try $alpha:

$ alpha=2+5
$ beta=$alpha*3
$ echo $beta
2+5*3

A sure sign of guesswork: “let’s try”. Don’t guess. Know.

Quaternary attempt

Use let for arithmetic assignment:

$ alpha=2+5
$ let beta=$alpha*3
$ echo $beta
17

Quinary attempt

The problem was that $alpha literally returned the string 2+5. What is 2+5*3? Let’s avoid $:

$ alpha=2+5
$ let beta=alpha*3
$ echo $beta
21

Senary attempt

To be consistent, and to force arithmetic evaluation every step of the way, use let in the first assignment:

$ let alpha=2+5
$ let beta=alpha*3
$ echo $beta
21

Rules

Let’s boil this down into rules:

• Use let for every arithmetic assignment:
  • let alpha=beta*34+delta
  • let gamma=42
• Avoid $ in arithmetic expressions.
• Use quotes to protect parentheses:
  • let alpha='(beta+gamma)*3'

Comedy of errors

$ alpha=1
$ beta=2             # no spaces around = 
$ gamma=alpha+beta
$ echo gamma
gamma
$ echo $gamma
alpha+beta
$ delta=$alpha+$beta
$ echo $delta
1+2
$ let epsilon=alpha+beta
$ echo $epsilon
3

Arrays

Somewhat limited, and certainly inelegant, capabilities compared to C:

$ array=(one two three)
$ echo ${array[1]}
two
$ echo ${array[*]}
one two three
$ echo ${#array[*]}
3

Filename Expressions

• used for matching filenames
• not very powerful:
  • * — matches anything of any length
  • [aeiou] — matches any one lower-case vowel
  • [a-z] — matches any one lower-case letter
• anchored:
  • The pattern a doesn’t match bonehead
  • The pattern *a* matches bonehead

Filename Expressions Examples

$ cd ~/bin
$ ls
checkin		      curve		  imv	p	 scores   wikidiff
checkin-checker       demo-script	  l	peek	 stats	  wikiedit
checkin-file-checker  domoss		  ll	playpen  tools	  wikigrep
checkin_prog	      e			  lsf	pwget	 u	  wikiupdate
chit		      grade		  moss	ruler	 unold	  wikiwhence
cls		      grade-busy	  new	run	 untar
code		      grade-file-checker  note	runner	 vman
cronedit	      grades		  old	save	 wikicat
$ ls ad
/bin/ls: cannot access 'ad': No such file or directory
$ ls *ad*
grade  grade-busy  grade-file-checker  grades
$ ls [check]in*
/bin/ls: cannot access '[check]in*': No such file or directory
$ ls checkin*
checkin  checkin-checker  checkin-file-checker	checkin_prog

Regular Expressions

• capabilities beyond many programming languages
• support pattern matching in grep and vi commands
• more powerful:
  • . — matches any single character
  • [0-9a-fA-F] — matches any one hexadecimal digit
  • [^a-zA-Z] — matches any single character that is not alphabetic
  • * — matches zero or more of what just came before
  • ^,$ — matches the beginning & end of a line
  • \d is [0-9], \w is [A-Za-z0-9_], \s is [ \f\t\n\r]
• not anchored:
  • The pattern a matches the line my dog has fleas

Levels of Regular Expressions

Beware of “levels” of regular expressions.

• grep — most basic (BRE, Basic Regular Expression)
• egrep — extended (ERE, Extended Regular Expression)
• grep -P — superior (PCRE, Perl Compatible Regular Expression)

Obsolete stuff

Use obsolete features only when dealing with obsolete computers.

if [ $PWD = / ]       # ☠ avoid single [ ]
if [[ $PWD = / ]]     # good

if [[ $n -eq 100 ]]   # ☠ avoid [[ … ]] for numbers
if (($n==100))        # better, but still poor
if ((n==100))         # good

now=`date`            # ☠ `backquotes` resemble 'quotes'
now=$(date)           # good

sum=`expr $sum + $n`  # ☠ invoking an external program‽
sum=$(expr $sum + $n) # ☠ not quite as bad
let sum=$sum+$n       # poor
let sum=sum+n         # ok
let sum+=n            # good

Script Example

#! /bin/bash
#
# Go into a temporary “playpen” directory; clean it up when done.

# If we can’t execute a file in TMPDIR, then change it to somewhere executable.
script=$(mktemp -t playpen-script-XXXXXX)
chmod u=rx,go= "$script"
"$script" 2>&- || TMPDIR=~/tmp
rm -f "$script"

cd "$(mktemp -d -t playpen-XXXXXX)"	# Create temporary dir.
cp -r ~/.playpen/* . 2>&-		# Files to play with
chmod -R u+rw .				# Works even if no files got copied.
ls -lhog | grep -v '^total '		# Show what’s here.
$SHELL
chmod -R u=rwx .			# Make everything removable.
cd /tmp					# Get away from temporary directory.
rm -rf $OLDPWD				# Remove previous, temporary, directory.