Preliminaries: Using the commandline, signing into a server, and submitting jobs to a HPC - barrettlab/2021-Genomics-bootcamp GitHub Wiki
How to sign into a server
1. The first thing you will need is a terminal application
A. Mac users, you have this already (Terminal). Finder > Terminal
B. Windows users, you have several options. I prefer MobaXterm, PuTTy, or GitBash.
PuTTy
MobaXterm -- has a nice little window built in!
GitBash
2. File transfer drag/drop applications:
FileZilla -- for any platform
WinSCP -- a nice application for Windows users
3. Text editors
bbedit -- for mac users. You may already have this with you OS distribution.
Notepad++ -- no, not the native "notepad" application!
4. OK, now you have your terminal application open. Now what? you should see the following:
[cfb0001.LSB5218] >
### or something like that...
Let's log into a server:
### General usage below. You will use YOUR username, my server address, etc.
ssh [email protected]
### Now, enter your case-sensitive password.
### Once you have successfully logged in, to may need to type:
bash
### This will enter you into the bash shell (before you might have just been in the Linux system but not in bash)
### to log off, type 'exit'
Logging into ‘myco’ (the Barrett Lab server) and a HPC cluster: this is a short video by Dhanu Ramachandran, our CPING postdoc.
Now, try out the following commands:
man
man ls
man grep
clear
history
date
whoami
id
groups
who
last
hostname
w
uname
ls
ls -l
ls -a
touch file1.txt
ls
cp file1.txt file2.txt
ls
mv file2.txt file3.txt
ls # to see what just happened -- how does it compare to 'cp'?
head file1.txt # this is a blank file!
echo "How's it going?" >> file1.txt
head file1.txt
cat file1.txt
echo "ATGGTGACGTGCAAACGTGTGTGTGTGTGTGTG" > file2.txt
head file2.txt
cat file1.txt file2.txt > file3.txt ; head file3.txt
less file3.txt # press 'q' to quit
top # list all current processes. This allows you to check who is running what. press 'q' to exit.
htop # A super fancy version of top. Use this to monitor RAM usage across multiple cores on a server. Press 'F10' button to exit
# kill ### DON'T enter this now, but this is how you stop a particular process
ln #make links between files
pwd
cd ..
ls
pwd
cd -
pwd
mkdir new_directory
ls new_directory
5. Some important places in the directory structure
pwd # where am I?
cd ~ # go to home folder
cd / # go to the ROOT of the directory structure
cd /home/cbarrett # The absolute path to your home folder. This always leads with the ROOT '/'
cd /usr/local/src # We typically use this folder to store source code before compiling new software. Download new software here
cd /usr/local/bin # This is where we keep our binaries, the compiled, executable software
cd /data # This is where we store everyones' data
cd /data/cbarrett # This is where I store my data
cd /data/cbarrett [press tab twice] # This will list all the subdirs of /data/cbarrett
6. SUDO -- this is a really important thing to do in linux, e.g. Ubuntu linux.
sudo # To see some options
sudo [some_command] # Enter your login password. This will give you permission to do things like modify files and run software # It is really just a security thing, giving you elevated permissions to run admin tasks
Some more information on sudo
7. Submitting jobs to a High Performance Computing cluster
Dhanu's GitHub site: HPC job submission
8. Setting up your $PATH: Telling UNIX where to find installed software
Two choices:
A. Simply copy & paste the code below
export PATH=/usr/local/bin:$PATH
export PATH=/usr/bin:$PATH
export PATH=/usr/local/src:$PATH
export PATH=/usr/local/src/ncbi-blast-2.10.1+/bin:$PATH
export PATH=/data/mmckain/Fast-Plast:$PATH
source ~/.bashrc
B. Edit your .bashrc file and paste in the text below, then save
sudo nano ~/.bashrc
### This is your bashrc profile, which tells UNIX where to look for software
export PATH=/usr/local/bin:$PATH
export PATH=/usr/bin:$PATH
export PATH=/usr/local/src:$PATH
export PATH=/usr/local/src/ncbi-blast-2.10.1+/bin:$PATH
export PATH=/data/mmckain/Fast-Plast:$PATH
Then type:
source ~/.bashrc
Save and exit:
ctrl+X
### To save and exit