Shell scripting

Task automation requires multiple instructions that have to run on demand. To combine multiple commands together for automation, we need to write a shell script.

Since Bash is the default shell on most Linux distributions, we will learn Bash scripting.

Although I recommend using the Fish shell when using the terminal and although Fish supports scripting too, I would not recommend using it for scripting since you would not be able to share and run these scripts anywhere. You need to have Fish installed which is not always the case.

I use Fish for my personal scripts. But if I write a script that will be shared with others, then I write it in Bash. Learn to write Bash scripts first before considering using Fish scripts, even for your personal scripts.

First Bash script

Let's write our first Bash script:

#!/usr/bin/bash

echo "What is your favorite operating system after reading this book?"
echo "1. Linux"
echo "2. Windows"
echo "3. Mac"

RIGHT_ANSWER=1

# `-p` Sets the prompt message
read -p "Enter a number: " ANSWER

if [ $ANSWER == $RIGHT_ANSWER ]
then
    echo "Good choice!"
else
    # Any answer other than 1
    echo "Nah, that can't be right! It must be an error!" 1>&2
fi

Copy this code into a file called which-os.sh.

Now, run chmod u+x which-os.sh. Then run ./which-os.sh. Don't worry, everything will be explained afterwards.

After running the script, you will see a prompt asking you to enter a number corresponding to an operating system. If you choose Linux, you get the output "Good choice". This output is in the standard output.

If you don't choose Linux, you get the output "Nah, that can't be right! (...)". This output is redirected to the standard error.

The building blocks of the script above will be explained in the next sections.

Shebang

The first line of our first script starts with #! which is called the shebang. The shebang is followed by the program that runs the script. Since the script is a Bash script, we use the program bash to run it.

But writing bash after the shebang is not enough. We have to specify the full path to the program. We can find out the path of a program by using the command which:

$ which bash
/usr/bin/bash

You can also write a Python script and add a shebang at its beginning. We can find out the path to the Python interpreter by running the following:

$ which python3
/usr/bin/python3

This means that we can now write this script:

#!/usr/bin/python3

print("Hello world!")

Let's save this tiny Python script as hello_world.py, make it executable with chmod (will be explained later) and then run it:

$ chmod u+x hello_world.py

$ ./hello_world.py
Hello world!

We could have written the Python script without the shebang, but then we would have to run with python3 hello_world.py. Adding the shebang lets you see a script as a program and ignore what language it is written in when running it.

You can use the shebang with any program that can run a script.

Variables

In our first Bash script, we have the line RIGHT_ANSWER=1. This line defines a variable with the name RIGHT_ANSWER and the value 1.

To define a variable in Bash, you have to write the name of the variable directly followed by an equal sign =. The equal sign has to be directly followed by the value of the variable.

directly followed by means that spaces between the variable name, the equal sign = and the value are not allowed!

But what if you want to set a variable equal to a string that contains spaces?

In that case, you have to use quotation marks ". For example:

HELLO="Hello world!"

To read the value of a defined variable, we use a dollar sign $ before the variable name. For example:

echo $HELLO

The line above would output Hello world!.

You can use defined variable inside a variable definition:

MESSAGE="Tux says: $HELLO"
echo $MESSAGE

The two lines above would lead to the output Tux says: Hello world!.

Capturing command output

You can capture the standard output of a command using the the syntax $(COMMAND). Example:

BASH_VERSION=$(bash --version)

The line above saves the output of the command bash --version in the variable BASH_VERSION.

Let's run the command in the terminal first to see its output:

$ bash --version
GNU bash, version 5.2.15(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Now, let's output the variable that we did define above:

$ echo $BASH_VERSION
GNU bash, version 5.2.15(1)-release (x86_64-redhat-linux-gnu) Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

You can see that the lines are squashed into one line! If you want to output the lines without them being squashed, you have to use quotation marks ":

$ echo "$BASH_VERSION"
GNU bash, version 5.1.16(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

This is the output that we expect 😃

Environment variables

Let's write the following tiny script hello.sh:

#!/usr/bin/bash

echo "Hello $USER!"

Now, we run the script. The output on my machine is:

$ chmod u+x hello.sh

$ ./hello.sh
Hello mo8it!

We see Hello mo8it! as output. This is because my user name on my machine is mo8it. USER is a so called environment variable that is defined for all programs. If you run the script on your machine, you will get your username instead of mo8it.

Now, let's run the following:

$ USER=Tux ./hello.sh
Hello Tux!

We defined an environment variable USER with the value Tux just before running the script. This variable overwrites the global value of the variable USER in our script. Therefore we get the output Hello Tux! and not our user name.

You can use environment variables not only to overwrite existing ones variables. These are basically variables that can be used by the program that you specify after their definition.

Comments

In our first Bash script, you can find three lines starting with a hashtag #. Two lines are comments, the shebang #! is an exception as a special comment at the beginning of a file that is not ignored while running the script. All other lines that start with # are comments that are ignored by the computer. Comments are only for humans to explain things.

Especially in long scripts, you should write comments to explain what the script itself and its "non trivial sections" do.

User input

In a script, you can ask for user input. To do so, you can use the command read.

In our first Bash script, we use read to ask the user for his answer. The input is then saved in the variable ANSWER (you can also choose a different name for this variable). After the line with read, you can use the variable storing the input just like any other variable.

Arguments

To read the n-th argument that is provided to a script, we can use $n.

Take a look at the following example script called arg.sh:

#!/usr/bin/bash

echo "The first argument is: $1"

When you run this script with an argument, you get the following output:

$ ./arg.sh "Hello"
The first argument is: Hello

Conditions

if block

Our first Bash script checks if the user input which is stored in the variable ANSWER equals the variable RIGHT_ANSWER which stores the value 1.

To check for a condition in Bash, we use the following syntax:

if [ CONDITION ]
then
    (...)
fi

Here, (...) stands for the commands that we want to run if the condition is true.

In our first Bash script, we check for equality of two variables with a double equal sign ==.

fi is not a typo! It is just if reversed to indicate the end of the if block. Although the syntax is not the best, you have to sadly accept it. Bash does not have the best syntax...

Speaking about syntax: You have to take spaces seriously with conditions.

For example, if we define the variable VAR=1, the following snippets do not work (or have an unexpected behavior):

1. No space after [

   if [$VAR == 1 ]
   then
       echo "VAR has the value 1"
   fi
   

2. No space before ]

   if [ $VAR == 1]
   then
       echo "VAR has the value 1"
   fi
   

3. No space before == but a space after ==

   if [ $VAR== 1 ]
   then
       echo "VAR has the value 1"
   fi
   

4. No space after == but a space before ==

   if [ $VAR ==1 ]
   then
       echo "VAR has the value 1"
   fi
   

5. No space before == and after ==

   if [ $VAR==1 ]
   then
       echo "VAR has the value 1"
   fi
   

But the following snippet works:

• Space after [, before ], before == and after ==

  if [ $VAR == 1 ]
  then
       echo "VAR has the value 1"
  fi
  

else block

The else block runs commands inside it only if the if condition is false. The syntax is:

if [ CONDITION ]
then
    # Runs only if CONDITION is true
    (...)
else
    # Runs only if CONDITION is false
    (...)
fi

Example:

if [ $VAR == 1 ]
then
    echo "VAR has the value 1"
else
    echo "VAR does not have the value 1"
fi