Getting familiar with the terminal and command lines - jniedzie/SVJanalysis_wiki GitHub Wiki

This page is a tutorial about terminal and command lines. If you've never heard about these terms, you are in the right place! 😃
Here is an introduction to basics, with direct application to perform useful tasks as we do every day!

The first task you have to do is to open a terminal. This depends on your OS.

Use the Windows Terminal. You can find all information here. If not installed by default on your machine, find relevant installation instructions on this page.

Search Terminal in your applications and click on the applications to open a terminal.

Search Terminal in your applications and click on the applications to open a terminal.

After you have requested an account on the PSI T3 (contact us if this hasn't been done), you should have received your user name. In the rest of these instructions, replace username by your user name.
You can log in onto the T3 machine with entering the following command-line in the terminal:

ssh -XY [email protected]

We will now learn about the basic and most useful command lines.

First, to check where you are on the machine, use pwd:

pwd

you should see:

/t3home/username

This is called your home directory. This is where you always end up when you log in.

You can check what is in your home directory by using ls:

ls

For now there should not be anything. So let's create a directory:

mkdir tutorials

and let's go inside this directory:

cd tutorials

Tip: You do not have to write down tutorials entirely! Start typing cd t and then hit Tab. The line will automatically be autocompleted to give cd tutorials! Autocompletion also works with command names, try mkdi, hit Tab once, nothing will happen, hit Tab once again to check it out.

Let's create another directory for specifically for this tutorial and go inside:

mkdir terminal
cd terminal

Great! Now where are we already? 😕 Yes, let's do pwd 😉, you should see

/t3home/username/tutorials/terminal

Let's check the content of the directory with ls, not surprisingly it's empty 😉. Let's create a file and check that it is in the current directory:

touch test.txt
ls

touch just creates an empty file if it does not exist, or "touches" (updates it without editing it) the file if it already exists. You can directly create and edit a file with vi, this will be explained below.

Make a copy of this file using cp:

cp test.txt test_copy.txt

and check that the new file exists using ls.

Actually, we don't like the name of that file, let's rename it using mv:

mv test_copy.txt test2.txt

and check that the file has been renamed using ls:

ls -lrth

Here we added the lrth option flags, to list file (l), ordered by time (t), in reversed order (r), and showing their size in a human readable format (h). This is very useful when you have a list of files in a directory (e.g. a list of config files), and know you need to update the last script.

Let's create a small Python script. Take a deep breath and type the following command line:

vi hello_world.py

Now type i, nothing will be written, but you will enter the insert mode (writing mode), notice the -- INSERT -- at the bottom left corner. You can now write things! Let's write this inside of this script:

print("Hello world!")

n = 10
for i in range(n):
    print("test: %d" %i)

You can copy this text and paste it by doing Ctrl + Shift + v in the terminal.
Now, exit the insert mode by hitting the escape key on your keyboard, usually labelled Esc.
And exit vi by typing ZZ (yes 2 capital "z").

Tip: There exists two categories of people in the programming world: those using vim (vi) and those using emacs. None is better than the other. You just have to make a choice, the right choice 😉. Here we will use vi, but feel free to use emacs if you prefer.

Tip: You can find a tutorial on vim by typing vimtutor

Let's check that the file exists. Look only at the solution if you don't remember how to do it!

ls

Now, we would like to run this script. For this we need Python to be installed. Let's check it:

which python

You should get as an answer /bin/python. Now let's run our code!

python hello_world.py

Now let's create a bash script hello_world.sh to run our script! Open a new file, write the bash shebang #!/bin/bash at the top of the file, and below write python hello_world.py.

1. Open the file with vi hello_world.sh
2. Hit i
3. Copy/write the shebang at the top of the file
4. Copy/write python hello_world.py below the shebang on a new line
5. Hit Esc
6. Hit ZZ to exit vi


The script should look like this:
#!/bin/bash
python hello_world.py

Now let's run the bash script. Before this we have to make executable:

chmod +x hello_world.sh

Now run the script:

./hello_world.sh

Tip: If you see -bash: ./test.sh: Permission denied it's because you did not make the script executable.

When you executed the python script above, you must have seen the following:

Hello world!
test: 0
test: 1
test: 2
test: 3
test: 4
test: 5
test: 6
test: 7
test: 8
test: 9

Let's redirect this into a log file:

./hello_world.sh > log.out 2>&1

This means that the output will be written to the log file log.out, and the error will be written to the same place.
You can check the content of the log file using cat:

cat log.out

Now what if we want to see only lines with test inside? Let's use grep:

cat log.out | grep "test"

The pipe | is used to combine commands. The output of the last command is passed as argument to the next one. This identical to doing:

grep "test" log.out

In case you want to see the line that do not start with test:

cat log.out | grep -Ev "^test"

Here we have used the regular expression (aka regex) ^test. The -E flag must be used to interpret the string as a regex. The -v flag means that we want to exclude the lines matching the regex inside of keeping them like done in the first example.

grep is a very powerful command line to select only some line from a file or the output of a command. You can read more about it online.

Another useful command is wc. With this command you can count lines, words or characters. This is very useful to count files, e.g. to check if the output directory of your code has all the files you expect. Here we can use it to count the number of lines with the string test in our log file:

cat log.out | grep "test" | wc -l

You should see 10 as output of this command.

A last useful command is echo. It just outputs what you give it as input. It is very useful to create a file for instance:

echo '#!/bin/bash' > hello_world_2.sh
echo 'echo Hello World!' >> hello_world_2.sh

Note that we used >> to append some more content to the file as > creates a new file each time.
Also Note that we used the ' quote instead of " such that the text in between the quotes is not interpreted by bash, else the exclamation mark ! would cause trouble! Check the content of the file and execute it. Yes, you already know how to do that 😉

cat hello_world_2.sh
chmod +x hello_world_2.sh
./hello_world_2.sh

On the T3, open ~/.bashrc (~ means your home directory) and add the following lines:

NC="\[\e[0m\]"
GREEN="\[\e[00;32m\]"
BLUE_BOLD="\[\e[01;34m\]"
export PS1="$GREEN[\\u@\\H \t] $BLUE_BOLD\w\n${NC}$ "

After that log out and in again. You should have a beautiful prompt 😃

We warmly recommend that you use an IDE to program, for instance VSCode. But knowing the basics of UNIX command line is a must for efficiently doing some operations otherwise cumbersome, like listing certain files to check their existence or number, or finding some information in a log file.