Bash Scripting!

The height of being lazy

Introduction

So this is the last section in this tutorial. Here we will introduce a concept called scripting.

This will be a brief introduction to Bash scripting. There is a lot more you can do but my aim here is to get you started and give you just enough that you can do useful work.

This section brings together a lot of what we learnt in previous sections (you'll see them referred to often). If some of this stuff doesn't really make sense, you may need to look back over previous sections and refresh your memory.

See our Bash Scripting Tutorial for a more comprehensive look into Bash Scripting.

So what are they?

A Bash script in computing terms is similar to a script in theatrical terms. It is a document stating what to say and do. Here, instead of the script being read and acted upon by a person, it is read and acted upon (or executed) by the computer.

A Bash script allows us to define a series of actions which the computer will then perform without us having to enter the commands ourselves. If a particular task is done often, or it is repetetive, then a script can be a useful tool.

A Bash script is interpreted (read and acted upon) by something called an interpreter. There are various interpreters on a typical linux system but we have been learning the Bash shell so we'll introduce bash scripts here.

Anything you can run on the command line you may place into a script and they will behave exactly the same. Vice versa, anything you can put into a script, you may run on the command line and again it will perform exactly the same.

The above statement is important to understand when creating scripts. When testing different parts of your script, as you're building it, it is often easiest to just run your commands directly on the command line.

A script is just a plain text file and it may have any name you like. You create them the same way you would any other text file, with just a plain old text editor (such as VI which we looked at in section 6).

A Simple Example

Below is a simple script. I recommend you create a similar file yourself and run it to get a feel for how they work. This script will print a message to the screen (using a program called echo) then give us a listing of what is in our current directory.

echo <message>

  1. cat myscript.sh
  2. #!/bin/bash
  3. # A simple demonstration script
  4. # Ryan 29/4/2017
  5.  
  6. echo Here are the files in your current directory:
  7. ls
  8. ls -l myscript.sh
  9. -rwxr-xr-x 1 ryan users 2 Jun 4 2012 myscript.sh
  10. ./myscript.sh
  11. Here are the files in your current directory:
  12. barry.txt bob example.png firstfile foo1 myoutput video.mpeg

Let's break it down:

  • Line 1 Let's start off by having a look at our script. Linux is an extensionless system so it is not required for scripts to have a .sh extension. It is common to put them on however to make them easy to identify.
  • Line 2 The very first line of a script should always be this line. This line identifies which interpreter should be used. The first two characters are referred to as a shebang. After that (important, no spaces) is the path to the interpreter.
  • Lines 3 and 4 Anything following a # is a comment. The interpreter will not run this, it is just here for our benefit. It is good practice to include your name, and the date you wrote the script as well as a one line quick description of what it does at the top of the script.
  • Line 6 We'll use a program called echo It will merely print whatever you place after it, as command line arguments, to the screen. Useful for printing messages.
  • Line 7 The next step of our script is to print the contents of our current directory.
  • Line 9 A script must have the execute permission before it may be run. Here I am just demonstrating that the file does have the right permissions..
  • Line 12 Now we run the script. I'll explain why we need the ./ a bit further down.
  • Lines 13 and 14 The output from running (or executing) our script.

Phew. A lot of important points were covered quite quickly there. Now let's have a look at them in more detail.

Important Points

The Shebang

The very first line of a script should tell the system which interpreter should be used on this file. It is important that this is the very first line of the script. It is also important that there are no spaces. The first two characters #! (the shebang) tell the system that directly after it will be a path to the interpreter to be used. If we don't know where our interpreter is located then we may use a program called which to find out.

which <program>

  1. which bash
  2. /bin/bash
  3. which ls
  4. /usr/bin/ls

If we leave this line out then our Bash script may still work. Most shells (bash included) will assume they are the interpreter if one is not specified. It is good practice to always include the interpreter however. Later on, you, or someone else, may run your script in conditions under which bash is not the shell currently in use and this could lead to undesireable outcomes.

The Name

Linux is an extensionless system. That means we may call our script whatever we like and it will not affect it's running in any way. While it is typical to put a .sh extension on our scripts, this is purely for convenience and is not required. We could name our script above simply myscript or even myscript.jpg and it would still run quite happily.

Comments

A comment is just a note in the script that does not get run, it is merely there for your benefit. Comments are easy to put in, all you need to do is place a hash ( # ) then anything after that is considered a comment. A comment can be a whole line or at the end of a line.

  1. cat myscript.sh
  2. #!/bin/bash
  3. # A comment which takes up a whole line
  4. ls # A comment at the end of the line

It is common practice to include a comment at the top of a script with a brief description of what the script does and also who wrote it and when. These are just basic things which people often wish to know about a script.

For the rest of the script, it is not necessary to comment every line. Most lines it will be self explanatory what they do. Only put comments in for important lines or to explain a particular command whose operation may not be immediately obvious.

Why the ./ ?

Linux is set up the way it is, largely for logical reasons. This peculiarity actually makes the system a bit safer for us. First a bit of background knowledge. When we type a command on the command line, the system runs through a preset series of directories, looking for the program we specified. We may find out these directories by looking at a particular variable PATH (more on these in the next section).

  1. echo $PATH
  2. /usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/usr/lib/mit/bin:/usr/lib/mit/sbin

The system will look in the first directory and if it finds the program it will run it, if not it will check the second directory and so on. Directories are separated by a colon ( : ).

The system will not look in any directories apart from these, it won't even look in your current directory. We can override this behaviour however by supplying a path. When we do so the system effectively says "Ah, you've told me where to look to find the script so I'll ignore the PATH and go straight to the location you've specified instead." You'll remember from section 2 (Basic Navigation) that a full stop ( . ) represents our current directory, so when we say ./myscript.sh we are actually tellling the system to look in our current directory to find the script. We could have used an absolute path as well ( /home/ryan/linuxtutorialwork/myscript.sh ) and it would have worked exactly the same, or a relative path if we are not currently in the same directory as the script ( ../linuxtutorialwork/myscript.sh ).

If it were possible to run scripts in your current directory without this mechanism then it would be easy, for instance, for someone to create a malicious script in a particular directory and name it ls or something similar. People would inadventently run it if they wanted to see what was in that directory.

Permissions

A script must have the execute permission before we may run it (even if we are the owner of the file). For safety reasons, you don't have execute permission by default so you have to add it. A good command to run to ensure your script is set up right is chmod 755 <script>.

Variables

A variable is a container for a simple piece of data. They are useful if we need to work out a particular thing and then use it later on. Variables are easy to set and refer to but they have a specific syntax that must be followed exactly for them to work.

  • When we set a variable, we specify it's name, followed directly by an equals sign ( = ) followed directly by the value. (So, no spaces on either side of the = sign.)
  • When we refer to a variable, we must place a dollar sign ( $ ) before the variable name.

A simple example.

  1. cat variableexample.sh
  2. #!/bin/bash
  3. # A simple demonstration of variables
  4. # Ryan 29/4/2017
  5.  
  6. name='Ryan'
  7. echo Hello $name
  8. ./variableexample.sh
  9. Hello Ryan

Command line arguments and More

When we run a script, there are several variables that get set automatically for us. Here are some of them:

  • $0 - The name of the script.
  • $1 - $9 - Any command line arguments given to the script. $1 is the first argument, $2 the second and so on.
  • $# - How many command line arguments were given to the script.
  • $* - All of the command line arguments.

There are other variables but these should be enough to get you going for now.

  1. cat morevariables.sh
  2. #!/bin/bash
  3. # A simple demonstration of variables
  4. # Ryan 29/4/2017
  5.  
  6. echo My name is $0 and I have been given $# command line arguments
  7. echo Here they are: $*
  8. echo And the 2nd command line argument is $2
  9. ./morevariables.sh bob fred sally
  10. My name is morevariables.sh and I have been given 3 command line arguments
  11. Here they are: bob fred sally
  12. And the 2nd command line argument is fred

Back ticks

It is also possible to save the output of a command to a variable and the mechanism we use for that is the backtick ( ` ) (Note it is a backtick not a single quote. Typically you'll find the backtick on the keybard to the left of the 1 (one) key.). Here is an example.

  1. cat backticks.sh
  2. #!/bin/bash
  3. # A simple demonstration of using backticks
  4. # Ryan 29/4/2017
  5.  
  6. lines=`cat $1 | wc -l`
  7. echo The number of lines in the file $1 is $lines
  8. ./backticks.sh testfile.txt
  9. The number of lines in the file testfile.txt is 12

A Sample Backup Script

Now let's put the stuff we've learnt so far into a script that actually does something useful. I keep all my projects in separate directories within a directory called projects in my home directory. I regularly take a backup of these projects and keep them in dated folders within a directory called projectbackups also in my home directory.

  1. cat projectbackup.sh
  2. #!/bin/bash
  3. # Backs up a single project directory
  4. # Ryan 29/4/2017
  5.  
  6. date=`date +%F`
  7. mkdir ~/projectbackups/$1_$date
  8. cp -R ~/projects/$1 ~/projectbackups/$1_$date
  9. echo Backup of $1 completed
  10. ./projectbackup.sh ocelot
  11. Backup of ocelot completed

You'll notice that I have used relative paths in the above script. By doing this I have made the script more generic. If one of my workmates wished to use it I could give them a copy and it would work just as well for them without modification. You should always think about making your scripts flexible and generic so they may easily be used by other users or adapted to similar situations. The more reusable your scripts are, the more time goes on, the less work you have to do :)

If Statements

So the above backup script makes my life a little easier, but what if I make a mistake? The script may fall over in a mess of error messages. In the example below I will introduce if statements. I'll only touch on them briefly. You should be able to work out their usage from the example and notes below. If you would like to know more then check out our Bash Scripting Tutorial which goes into much more detail.

(If this all seems too confusing, don't worry too much. Even with just the knowledge above you can still write quite useful and practical scripts to make your life easier.)

  1. cat projectbackup.sh
  2. #!/bin/bash
  3. # Backs up a single project directory
  4. # Ryan 29/4/2017
  5.  
  6. if [ $# != 1 ]
  7. then
  8.     echo Usage: A single argument which is the directory to backup
  9.     exit
  10. fi
  11. if [ ! -d ~/projects/$1 ]
  12. then
  13.     echo 'The given directory does not seem to exist (possible typo?)'
  14.     exit
  15. fi
  16. date=`date +%F`
  17.  
  18. # Do we already have a backup folder for todays date?
  19. if [ -d ~/projectbackups/$1_$date ]
  20. then
  21.     echo 'This project has already been backed up today, overwrite?'
  22.     read answer
  23.     if [ $answer != 'y' ]
  24.     then
  25.         exit
  26.     fi
  27. else
  28.     mkdir ~/projectbackups/$1_$date
  29. fi
  30. cp -R ~/projects/$1 ~/projectbackups/$1_$date
  31. echo Backup of $1 completed

Let's break it down:

  • Line 6 Our first if statement. The formatting is important. Note where the spaces are as they are required for it to work properly. In this statement we are asking if the number of arguments ( $# ) is not equal to ( != ) one.
  • Line 8 If not then the script has not been properly invoked. Print a message explaining how it should be used.
  • Line 9 Because the script has not been invoked properly we wish to exit the script before going any further.
  • Line 10 To indicate the end of an if statement we have a single line which has fi (if backwards) on it.
  • Line 11 If statements can test a lot of different things. Here the exclamation mark ( ! ) means not, the -d means 'the path exists and is a directory'. So the line reads as 'If the given directory does not exist'
  • Line 22 It is possible to ask the user for input. The command we use for that is read. read takes a single argument which is the variable to store the answer in.
  • Line 23 Let's see how the user responded and act accordingly.

You'll notice that certain lines are indented in the above code. This is not necessary but is generally considered good practice as it makes the code a lot easier to read.

If statements actually make use of a command called test. If you would like to know all the different comparisons you may perform then have a look at the manual page for test.

This has been a very brief introduction to Bash Scripting. See our Bash Scripting Tutorial for a more comprehensive look into Bash Scripting.

Summary

#!
Shebang. Indicates which interpreter a script should be run with.
echo
Print a message to the screen.
which
Tells you the path to a particular program.
$
Placed before a variable name when we are referring to it's value.
` `
Backticks. Used to save the output of a program into a variable.
date
Prints the date.
if [ ] then else fi
Perform basic conditional logic.
Behaves the same
Anything you may do on the command line you may do in a script and it will behave exactly the same.
Formatting
Bash scripts are particularly picky when it comes to formatting. Make sure spaces are put where they are needed and not put when they are not needed.

Activities

Let's automate:

To solve these activities you'll need to bring together your skills and knowledge from this section and all the previous sections.

  • First off, think about writing your own backup script. You can make it as simple or complex as you like. Maybe start off with a really simple one and progressively improve it.
  • Now see if you can write a script that will give you a report about a given directory. Things you could report on include
    • How many files are in the directory?
    • How many directories are in the directory?
    • What is the biggest file?
    • What is the most recently modified or created file?
    • A list of people who own files in the directory.
    • Anything else you can think of.
    .