Takeout: Usefull Tools - ramirezfranciscof/aiida-core GitHub Wiki

Checking the space on disk

The regular df -h command shows a lot of unnecessary files: the /dev/loop* and the tmpfs:

In Unix-like operating systems, a loop device, vnd (vnode disk), or lofi (loop file interface) is a pseudo-device that makes a computer file accessible as a block device. Block special files or block devices provide buffered access to hardware devices, and provide some abstraction from their specifics. Read more here.
Tmpfs is a file system which keeps all of its files in virtual memory. Everything in tmpfs is temporary in the sense that no files will be created on your hard drive. If you unmount a tmpfs instance, everything stored therein is lost. The tmpfs puts everything into the kernel internal caches and grows and shrinks to accommodate the files it contains and is able to swap unneeded pages out to swap space. It has maximum size limits which can be adjusted on the fly via ‘mount -o remount …’. If you compare it to ramfs (which was the template to create tmpfs) you gain swapping and limit checking. Another similar thing is the RAM disk (/dev/ram*), which simulates a fixed size hard disk in physical RAM, where you have to create an ordinary filesystem on top. Ramdisks cannot swap and you do not have the possibility to resize them.

df -h -x squashfs -x tmpfs

# Also this, but I don't know about devtmpfs
df -h -x squashfs -x tmpfs -x devtmpfs

# Convenient to add to bashrc:
# alias mydf='df -h -x squashfs -x tmpfs -x devtmpfs'

Source: https://www.cyberciti.biz/faq/linux-check-disk-space-command/ Source: https://clay-atlas.com/us/blog/2021/07/03/linux-en-hide-dev-loop-display/

Plotting in the terminal with gnuplotlib

Source: https://github.com/dkogan/gnuplotlib

Pip config & tools

(1) For getting information on which versions of software are being used (local, from PyPI, etc.)

$ pip list
Package                      Version      Location                 
---------------------------- ------------ -------------------------
aiida-core                   1.0.0        /home/framirez/aiida-fork
aiida-export-migration-tests 0.7.0        
alabaster                    0.7.12       
aldjemy                      0.9.1        
alembic                      1.0.7        
aniso8601                    7.0.0

Screen

You want to be using GNU Screen. It is super awesome!

ssh [email protected]
screen               #start a screen session
(run-a-long-process)

CTRL+a , d to detatch from your screen session

exit                 #disconnect from the server, while run-a-long-process continues

When you come back to your laptop:

ssh [email protected]
screen -r            #resume the screen session

Then check out the progress of your long-running process!

screen is a very comprehensive tool, and can do a lot more than what I've described. While in a screen session, try ctrl+a,? to learn a few common commands. Probably the most common are:

CTRL+a , c to create a new window
CTRL+a , n to switch to the next window in your screen session
CTRL+a , p to switch to the previous window in your screen session
if you log in from a bunch of different systems, you may have accidentally left yourself attached to an active screen session on a different computer. for that reason, I always resume with screen -d -r to ensure that if another shell is attached to my screen session, it will be detached before I resume it on my current system.

It might be annoying to scroll, you need to do something special: use CTRL+a, Escape to enter copy mode: now you can move using the arrows. When you're done, hit q or Escape to get back to the end of the scroll buffer (source).

Naming your screen sessions

You can name a session when starting it with the -S <name> option. From within a running screen, you can change it by typing

Ctrl+A,: followed by sessionname <name>.

You can view running screen sessions with screen -ls, and connect to one by name with

screen -xS name

Running in background

You can send an already running foreground job to background as explained below:

Press ‘CTRL+Z’ which will suspend the current foreground job.
Execute bg to make that command to execute in background.

For example, if you’ve forgot to execute a job in a background, you don’t need to kill the current job and start a new background job. Instead, suspend the current job and put it in the background as shown below.

$ find / -ctime -1 > /tmp/changed-file-list.txt
$ [CTRL-Z]
[2]+  Stopped                 find / -ctime -1 > /tmp/changed-file-list.txt
$ bg

You can list out the background jobs with the command jobs. Sample output of jobs command is

$ jobs
[1]   Running                 bash download-file.sh &
[2]-  Running                 evolution &
[3]+  Done                    nautilus .

You can bring a background job to the foreground using fg command. When executed without arguments, it will take the most recent background job to the foreground.

# fg

If you have multiple background ground jobs, and would want to bring a certain job to the foreground, execute jobs command which will show the job id and command.

In the following example, fg %1 will bring the job#1 (i.e download-file.sh) to the foreground.

$ jobs
[1]   Running                 bash download-file.sh &
[2]-  Running                 evolution &
[3]+  Done                    nautilus .
$ fg %1

If you want to kill a specific background job use, kill %job-number. For example, to kill the job 2 use $ kill %2.