This page includes a non-exhaustive list of shell commands that you might find useful. It was originally compiled by Michael Mangus, but has been added to over the years. If there is something you are trying to do in a terminal shell, this is a great place to start looking for tips.

$ ctrl+c

Tmux is a great resource. It keeps a terminal session active, even if you flip the lid closed on your laptop or close your terminal (doing these things will kill your terminal session, without a tmux session). This is great if you are trying to run an analysis on a remote machine. Also lets you set up a session and keep it configured just the way you like it. There are a number of ways you can customize a tmux session, but the basic commands to start one are listed below:

After logging into a remote machine via ssh (e.g., $ ssh username@hostname)

Start a new tmux session. NB, as a convention, Richard usually replaces sessionname with his initials (rwh) so that he doesn't forget what the sessionname is:

$ tmux new-session -t sessionname

If you close your terminal, or your laptop lid, you can always reattach to the tmux section by first sshing into the remote machine (e.g., $ ssh username@hostname) and then running this command (being sure to use the same sessioname as you used above when starting the tmux session:

$ tmux attach -t sessionname

Check that this works! Close the terminal, and then ssh back into the remote machine, and then see if you can get back to your tmux session.

The tmux session will persist until you (a) turn off the machine where you initiated the session (e.g., the remote machine), or (b) kill the session. To kill a session, enter:

$ tmux kill-session -t sessionname

Sometimes it is really helpful to have two panes in the same tmux session. To try that, follow these steps:

1. On our keyboard, press ctrl+b (this is a prefix that you need to enter before running additional commands)
2. Release ctrl + b and then press the % key on your keyboard
3. Woah! A new pane on the right! But, how do I get my cursor to the pane on the left? Press ctrl + b and then, after releasing, press the left arrow key (or, if you want to move from the left pane to the right, follow those steps, but pressing the right arrow key)
4. How do I kill the current pane I'm working in? Simple, press ctrl + b and then, after releasing, press the x key. You will be prompted with kill-pane #? (y/n). Press the y key to kill the pane.
5. There are all sorts of cool things you can do with tmux, read a cheat sheet

If you want very detailed information about system load, all the processes running, their PID, etc, run:

$ top

That said, the top command consumes system resources. So, if you are running a big analysis, this might not be a great idea. If you want a quick look at load averages, then run:

$ uptime

By default, you can only read the first ten lines of a file. You can change the number of lines displayed by specifying a number option.

$ head -20 filename

You can do the same for the last n lines of a file.

$ tail -20 filename

You will connect using the secure shell protocol, SSH. SSH encrypts all traffic and requires authentication. We are not running an FTP server; if you need to transfer files, use the secure copy command, scp (see explanation below). To ssh into a system, just open a terminal and execute:

$ ssh username@machineipaddress

NOTE: You will need to be connected to the VPN to ssh into remote machines (unless you are doing this from a lab workstation).

You will be prompted for your password. This can get annoying after a while. The better solution is to use cryptographic keypair. Authentication for these servers is handled by using cryptographic keypairs. Your public key is stored on the server. The public key matches exactly one private key, which is stored on your laptop and should never be shared with anyone. When you attempt to login, the server checks whether your private key on your personal computer matches one of the public keys for your user on the server. You will not be prompted for a password.

This assumes you have never generated an RSA keypair for your system. If you have, you can skip to step 3.

1. Create the RSA keypair

   $ ssh-keygen -t rsa

2. Store the Keys and Passphrase

   Once you have entered the Gen Key command, you will get a few more questions:

   Enter file in which to save the key (/home/username/.ssh/id_rsa):

   You can press enter here, saving the file to the user home directory

   Enter passphrase (empty for no passphrase):

   You do not have to enter a passprhase, but you REALLY should. This is an extra password on top of your key and it makes your system much more secure and limits people's access. If you do NOT want to enter a passphrase, just press "enter" on your keyboard. You will be prompted to enter the passprhase again. If you did not enter a passphrase, just hit "enter" again on your keyboard. You'll get an output telling you what happened.

   The public key is now located in /home/username/.ssh/id_rsa.pub. The private key (identification) is now located in /home/username/.ssh/id_rsa.

3. Copy the public key

   Now it is time to place your public key on the remote machine. You'll want to do this by copying your public key to the remote machine's authorized_keys file. This is simple:

   $ ssh-copy-id username@machineipaddress

   You will get a message asking you if you want to add the key. Type yes

   You can now ssh into a system without entering your password.

4. Test that it works. Just open a terminal and execute:

$ ssh username@machineipaddress

So long as you don't get promoted for your password, it worked!

If you get stuck, here is a great guide

It can get pretty difficult to remember a remote machine's IP address. So, you can give the remote machine a shortname. This makes it easier to ssh into a remote machine. This is simple:

1. Change to the /etc directory

   $ cd /etc

2. Use a text editor to add IP address information. This example uses the nano text editor.

   $ sudo nano hosts

3. Add the IP address(addresses) for the lab workstations (remote machines). You'll want to add these IPs BELOW the IP address(addresses) for your local machine (probably something like 127.0.0.1). You'll want to add these IP addresses on a new line at the VERY BOTTOM of the "hosts" file. IP addresses for lab systems are listed below, and the workstation name is the "hostname". Put the IP address and "hostname" for your workstation in the "hosts" file. Alternatively, you can put all in, if you plan to use all workstations. NB, the space between the IP address and the hostname is a tab:

169.237.107.49	miller
169.237.107.47	corkin
169.237.107.48	dennett

4. check that it worked by opening a terminal and entering this command (being sure to replace yourusername with your username and yourhostname with the hostname for the workstation you are trying to loginto)

$ ssh yourusername@yourhostname

NB: NB marr is different, see below

A ssh alias makes it even faster to connect to a remote machine. To set one up:

1. Edit the hidden .bash_aliases file in your home directoury. This example uses the nano text editor.

   $ nano ~/.bash_aliases

2. In the file, write a new line for each alias, e.g.,

   alias corkin='ssh localusernameoncorkin@corkin'
   alias marr=username@peloton\.cse\.ucdavis\.edu'
   

3. Restart .bash_aliases

   $ source ~/.bash_aliases

4. You can now ssh into systems simply by entering, e.g.,

   $ marr

Now that you have this set up, lets say you just did an analysis on Unity, and you want to have a look at the results. You could copy the data from Unity to your local workstation, but that is slow and a pain. Or, you could use x-forwarding to run some simple graphics programs that rely on x.

The first step is to ssh into Unity with x-forwarding

$ ssh -Xc scom-marr

NOTE: the -X flag starts x-forwarding for your ssh session.

You can now call programs that rely on x. For example, fsl (and its various viewers):

$ fsl

The text-based environment you'll use to interact with remote machines is called a shell. There are many different programs. By default, Ubuntu and macOS use the bash shell. The next section explains some core shell concepts before providing a non-exhaustive list of shell commands.

You can use wildcards to make the same command happen to multiple files (for example). There are all sorts of ways to use wildcards, and there are several different types of wildcards. These can be very powerful, but also very dangerous. The most common wildcard you will use is probably *.

* can represent any character, or even no character. This can be helpful if you, say, want to list all dicom files in a directory, but not other non-dicom files in that directory. An exampele would be:

$ ls -lh *dcm

See here for a helpful list of ways to use wildcards.

Lets say you are changing a directory (more on that below). You do not have to type out the whole path. Instead, you can press the "tab" key on your keyboard to complete parts of the path.

Your input to the shell takes the general form:

$ command -flags arguments

where command is some program installed on the system, -flags are single letters which modify command behavior, and arguments are one or more objects to be accessed or modified by the command. Flags are generally needed for specialized uses of a command. Arguments are not required for all commands, but when they are, they are most often the paths to files. For example:

$ ls -lh /home/username

ls is a command to list files in a directory (see below) -lh tells the ls command to include more detailed information and use human-readable filesizes (see below) /home/username tells ls which directory to list files for, in this case, files in the home directory for username

NOTE: for certain programs, some flags are introduced with -- instead of - and are written out in words instead of individual letters.

All paths are interpreted relative to the current working directory, where all directories branch from the filesystem root, /. If you are working in /home/username, running command x and command /x are not equivalent. In this example, command x runs a program in your current working directory whereas command /x runs a prgram in the /x directory.

As an alternate example... If you are working in /home/username command x and command /home/username/x are equivalent.

0. man [command] – MANual for the given command. This is the 0th on the list because it is the most useful command out there! If you're ever confused about how to use some command (lets say chown, which I mention briefly below) then just do something like:

   $ man chown

1. pwd -- Print Working Directory i.e. return the path of the current directory. Doesn't take an argument.

2. ls (-lh) [directory] -- LiSt contents i.e. return a list of all files and directories inside the specified directory. If no directory is specified, it uses the current directory (equivalent to running ls ., see below). -l (Long) gives info on ownership, permissions, file size, and modification time. -h (Human-readable) prints filesizes with MB/GB/TB as appropriate for that file (default behavior doesn't vary the unit of measurement).

3. cd [directory] -- Change Directory to the one specified. Recalling that paths are interpreted based on the working directory, note that cd /etc and cd etc are different (unless you are currently working in /, which you generally shouldn't).

4. cp (-r) [file1] [file2] --CoPy file1 to file2. If file1 is a directory, you must specify -r (recursive) to copy all of its contents.

5. mv [file1] [file2] --MoVe file1 to file2. If you need to rename a file, use mv, as in:

   $ mv old_name new_name

6. df (-h) and du (-h) -- No idea what these are supposed to stand for, but you can list the filesystem usage (% of space used, etc) for all mounted filesystems using df -h. You can also calculate the size of a directory particular and all its children using du -h [directory].

7. top – Shows stats about memory and CPU usage, and the programs which are consuming the most resources. Doesn't take an argument.

8. locate [keyword] -- Used to locate a file within the filesystem whose location is unknown. If you have just installed something, it will not be indexed for searching yet; run 'sudo updatedb' to index it.

9. ps (-ef) -- ProceSes i.e. returns the processes currently running on the system which are owned by your user. -e returns processes owned by all users, while -f returns more detailed information. I suggest using -ef to get the most relevant information. Doesn't take an argument. Most useful in conjunction with grep, explained next.

10. grep [keyword] [file1] -- search file1 for keyword and return any matching lines. Specifying a file is optional; more often, grep's input is a pipe from another command (see details on how to use pipes below), as in:

    $ ps -ef | grep python

    This line will (a) run ps -ef and (b) use the output of ps -ef as input to grep, searching the list of all active processes for one with 'apache' as part of the name.

11. | -- The above is just one exampe of using a pipe | command. You can make complex arguments. For example, lets say you wanted to count the number of dicom files in the directory you are working in. You could do this with:

    $ ls -l | grep "dcm" | wc -l

    Here, ls -l lists all files in the working directory, grep "dcm" searches that output for any that include dcm, and wc -l` counts the number of lines. Together, this would tell you how many dicoms are in a working directory.

12. rm [file1] -- ReMove the specified file. THERE IS NO "TRASH." IF YOU USE THIS COMMAND RECKLESSLY, THINGS WILL BREAK IN VERY UGLY WAYS. READ THE PARANOIA SECTION BELOW BEFORE YOU USE THIS COMMAND!

13. touch <filename> -- Use this to create a new empty file. For example:

    $ touch deletme.txt

All files stored on the system have access privileges (“permissions”) specifying who has access to read, write, or execute the file. You can see these by using ls -lh. For example, consider this output from a file named library.txt that I have saved:

rwxr-xr-x 1 huskey_admin huskey_admin 246k Nov 13 2016 library.txt

Permissions depend on the owner (a particular user) and the group (a group of users). These are the 3rd and 4th columns in the output from ls -lh. Here, we can see the file is owned by the user huskey_admin and the group huskey_admin (all users are part of a group named after their username which is the default group ownership for their personal files).

The first column of r's, -'s, w's and x's shows the permissions. The first character represents that this file is a not directory (d if it is, - if not). The next set of three characters shows that file is Readable, Writeable, and Executable by its owner (huskey_admin). The next set of three shows that it is readable and executable, but not writeable, by other members of the specified group (also huskey_admin). The final set of three shows that it is readable and executable, but not writeable, by "others" – that is, users who are neither the owner nor a member of the group.

If you need to change ownership, use the command chown [user]:[group] [file] as in:

$ chown nobody:nogroup library.txt

To change user ownership:

$ chown nobody library.txt

If you need to change permissions, use the command chmod [permissions] [file] as in:

chmod u+x script.py

See 'man chmod' and 'man chown' for details on how to use these commands properly.

If you need to expand premissions for directory you didn’t create:

sudo chmod a+rw somefile

NOTE: If you are trying to change permissions or ownership of a directory, you will need to use the -R flag.

You may also find this guide helpful.

Ubuntu (and other Debian-type Linux distributions) do not have a root user account. Instead, root-level privileges are granted by use of the sudo (Super-User DO) command, which takes another command as its argument, as in:

$ sudo nano /etc/some.config

A user can run sudo only if that user is a member of the group sudo (new lab members are not). When you run a command with sudo for the first time in a session, you will be prompted for your password. There is no separate sudo password – just the regular password for your account. Subsequent uses of 'sudo' in the same session won't prompt again, so don't count on the password prompt as a safety net against doing something bad.

There are some beefy command-line text editors, but they have a steep learning curve. If you want to upgrade your computer skills, I suggest you learn to use emacs. Others will virulently object that you should learn vi instead of emacs. Honestly, though, if you just want to edit a configruation file, use nano, as in:

$ nano /etc/example.cfg

Nano is the most self-explanatory and user-friendly editor available. The relevant commands are listed at the bottom of the screen. Read ^ as the control key. Note that “Write Out” = save.

1. If you want to run two commands serially, use &&, as in:

   $ cd /home/ysername && ls -lh

2. If you need to embed a command as an argument to a second command, enclose the embedded command inside a pair of grave accent marks (also known as backticks), as in:

   $ ls -lh `locate authorized_keys`

3. The current directory is represented as '.' This is sometimes useful for commands like cp. For instance, if you want to copy a default configuration file from /etc/ to your current directory:

   $ cp /etc/default/example.cfg .

You also need this shortcut if you want to execute a file that is in the current local directory rather than one of the main application directories (defined in your users PATH environment variable, which you can check with echo $PATH). For instance, if you're in your home folder and want to run an executable file /home/username/script.py, use: $ ./script.py

4. The directory 1 level higher in the filesystem hierarchy is represented as '..' If you want to change your directory to 1 level higher (e.g. from /home/username/downloads to /home/username), use:

   cd ..

5. Your home directory is represented as '~'. You can use this to access any files/subdirectories of your home as well, e.g. ~/downloads/funny.gif If you want to reference another user's home, use ~username.

6. If you want to repeat the most recent command, use '!!'. For example, if a command failed because you didn't have sufficient privileges and you want to run it again as a superuser:

   sudo !!

7. Ways to find files. Example one is for a directory, the second is more general

   find / -type d -name directoryname

   locate directoryorfilename

8. How to restart a system remotely. Do not do this unless absolutely necessary. We do not turn systems off. If you are going to reboot a system, check with lab members first to make sure they aren't also using it.

   Restart now:

   $ sudo /sbin/shutdown -r now

   Restart in 2 minutes with a warning message:

   $ sudo /sbin/shutdown -r +2 system patch and restart

   Shut a system down, NO RESTART, in 2 minutes with a warning message

   $ sudo /sbin/shutdown -h +2 system shut down for upcoming power outage

9. If a machine gets into a login loop on the lock screen, try this:

   $ sudo dpkg-reconfigure gnome-shell

10. If you get stuck on an Authentication Error loop on the lock screen, move to a TTY with ctrl+alt+f2 or ctrl+alt+f3 on the lock screen. Login and:

    $ sudo apt install --reinstall gdm3 gnome-settings-daemon gnome-shell ubuntu-desktop

    Then get back into the GUI session with ctrl+alt+f1

11. If the above does not work, try this:

    $ sudo nano /etc/sysctl.d/99-sysctl.conf

    Scroll to the bottom and add:

    $ fs.inotify.max_user_watches=1048576

    Restart.

"Unix shell" is an antonym for "idiot-proof" -- it is extremely unforgiving of mistakes. Please do not run any command that you don't understand. Please notify Richard of any changes you make to system configuration files (for instance, anything in /etc/).

Please be very careful using rm especially; there is no "trash" to restore files from. We keep a full backup, but there is a chance that you delete everything before an image the latest image has been captured, and then work is lost. Also, restoring from a backup is a giant pain. Be VERY careful when using rm.

Some simple rules for how to use rm responsibly:

1. Make sure you know which machine you're using: The default prompt in your shell shows is of the format username@computername [active directory]. So, if you see "usernamescom-corkin ~", you know you're in your home folder on Corkin. You can also check using the hostname command.

2. Make absolutely certain you have the correct path: Paths are relative to the current directory (pwd to check). Remember that the base of the filesystem is just the root directory, /. If you are in /home/username, running rm -rf x and rm -rf /x are NOT equivalent -- the former deletes /home/rwh/x, the latter /x.

3. Make absolutely certain you understand which user the command will be run as: When you use sudo, you have read/write access to every file on the system, which means (to reiterate) that you can mess everything up. Be absolutely certain that you know what is about to happen before running sudo rm!

4. Double-check any wildcards: Wildcards can help, but can also get you into serious trouble. Make sure you know exactly what will happen before you use them, especially with rm

In order to transfer a file from your personal machine to the remote machine, you need to use scp instead of connecting with ssh. So, open a new terminal window on your personal machine, and run something like this (which would transfer a file from your local machine to scom-corkin)

$ scp [file1] username@scom-corkin:

Note the : at the end of this command! If you leave it out, you'll find you have a new local copy of file1, named username@scom-corkin :). The file will be transferred to your home folder. If you want to copy [file1] from your local machine to a subdirectory of your home folder on the remote machine, use a command like:

$ scp [file1] username@scom-corkin:subdirectory

Or, if you want to copy something from a remote machine to your local machine, just flip the arguments:

$ scp username@scom-corkin:subdirectory/file1 [file1]

NOTE: Just like cp, scp needs the -r flag to copy a directory.

If you want to transfer directories from one remote machine to another remote machine, or you want to syncronize files on two machines, you are best off using rsync.

In this case, the example below assumes you want a subdirectory on scom-miller copied to your home directory on scom-corkin.

1. ssh into one of the remote machines (in this case, scom-corkin)

   $ scom-corkin

2. Change to the directory you want data copied INTO from the remote machine

   $ cd ~

3. Enter this command:

   $ rsync -rhvzlP username@scom-miller:home/username/subdirectory_i_want_copied/ .

   Note the final period. The above command transfers the CONTENTS OF THE subdirectory_i_want_copied directory on scom-miller to whatever directory you are working in on scom-corkin (in this instance, ~). If you want to transfer a directory itself, omit the "/" at the end of the path.

The flags are:

-r recurse through directories -h make sizes more easily readable -v be verbose -z compress the transfer (NOTE: lots of overhead, can really slow a transfer, may only be useful on slower internet connections) -l transfers symlinks (this is a lowercase l) -P use partial+progress options (lets you easily watch progress and resume if the connection is interrupted)

You can interrupt with ctl+c and restart it with the --append-verify option included without losing any progress.

More details on how to restart rsync.

NB: If you are sending data to/from Peloton, you'll need to add the following flag -e "ssh -p 2022".

Have fun with this one. Here is a helpful guide.

See also

Linux systems are less likely to get a virus, but there still is a risk. You should probably use a virus scanner. A good lightweight and free tool is Sophos

To get started:

1. Download the install package

2. gunzip the package

$ gunzip sav-linux-free-9.tgz

3. Untar the package

$ tar -xvf sav-linux-free-9.tar

4. Move into the newly uncompressed directory, run the installer, and follow the prompts

$ ./install.sh

Helpful tips:

1. To scan the entire system

$ sudo savscan /

2. To update the virus library

$ sudo /opt/sophos-av/bin/savupdate -v5

3. To check the version

$ sudo /opt/sophos-av/bin/savdstatus --version

For more tips, see the user guide and cheat sheet