shellsheet - feliyur/exercises GitHub Wiki
| Action | Syntax |
|---|---|
| Location of config/permissions on Ubuntu |
/etc/sudoers, /etc/groups
|
| Show storage devices and UUID | lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,UUID |
| Execute monitoring command every n seconds | watch -n <sec> <command> <command args> |
| Compute directory size, show filesystem info. |
du, df, ncdu
|
| Get / change hostname |
hostname to get, sudo hostnamectl set-hostname to set (or sudo edit /etc/hostname) |
| Get ip address |
ifconfig -a old style. New: ip address or ip a. |
| Who is logged on |
who -H, who -u, w, users, ps au the latter shows processes for each user. |
| Processes info | ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head |
| Copy symlink | cp -R --preserve=links |
| Clean up system logs | sudo journalctl --vacuum-size=100M |
| Find and delete empty directories | find /path/to/directory -type d -empty -delete |
| Find and filter files | find /path/to/directory -type [f,d] -name 'supports wildcards' [-maxdepth <N> -mindepth <M>] |
| More complex find usage example with exec |
find /path/to/directory -type d -exec chmod 770 {} \; -type f -exec chmod 660 {} \;. Mind the \; which ends the command, and {} which is placeholder of current item. |
| List contents recursively | tree [-L <max depth>] |
| Set a timeout for a blocking command |
timeout <n_seconds> <command and arguments> e.g. timeout 100 ssh <host> will close the ssh connection after 100 seconds. |
| Format a partition to ext4 | sudo mkfs.ext4 /dev/nvme0n1 |
| Set execute permissions only to subdirectories, but not to files | chmod -R rwX <target dir> |
| get and control file access lists | getfacl / setfacl |
| List groups user is part of |
id -a, groups
|
| List path and permissions to target | namei -l <target> |
| List open ports | sudo lsof -i -P -n | grep LISTEN |
| Manage service process | sudo systemctl [enable,disable,stop,start] <servicename>.service |
To search back in shell command history use Ctrl+r (this is called "reverse-i-search"). Can use Ctrl+s to search forward, but possibly need to remap an overshadowing shortcut for this key combination ("stop terminal sequence"), adding stty stop ^J to .bashrc (see here).
!<command prefix> executes last command starting with prefix. history shows command history.
Safe fstab line:
UUID=<device UUID> <mountpoint> none <options> 0 0
Mountpoint must exist. Can check UUID with lsblk
Worth to almost always use nofail to allow the system to boot even if the drive cannot be mounted. E.g. replace <options> with defaults,nofail.
For network-facing systems, worth to use safe options nosuid,nodev.
Mount ntfs filesystem:
UUID=F4E2C7FAE2C7BF5C /media/Windows ntfs defaults,nofail,nls=utf8,umask=000,dmask=027,fmask=137,uid=1000,gid=1000,windows_names 0 0
Can use lsblk -o NAME,MOUNTPOINT,SIZE,FSTYPE,UUID command to list drives and found out UUID of an unmounted drive.
| Functionality | Command |
|---|---|
| Create new user | sudo adduser [username] |
| Add user to group | sudo usermod -a -G groupName userName |
To hide a user from login screen see here:
sudo vim /var/lib/AccountsService/users/[username]Add / edit the following two lines:
[User]
SystemAccount=true
| Command | Function |
|---|---|
ssh <server> -C '<command>' |
Run command remotely and disconnect. |
ssh <server> -L<local-port>:localhost:<remote-port> |
maps local port to remote |
ssh <server> -R<local-port>:localhost:<remote-port> |
maps remote port to local (reverse tunnel) |
ssh <params> -N |
Do not execute a remote command (useful for port forwarding). |
Running ssh in the background (e.g. port-forwarding).
Installing: sudo apt-get install openssh-server
| Action | Command |
|---|---|
| Generate ssh key |
ssh-keygen -t ed25519 -C "[email protected]" Or ssh-keygen -t rsa -b 4096 -C "[email protected]"
|
| Add local ssh key to remote authorized_keys |
ssh-copy-id <username@remote>. On Windows: type <key .pub file> | ssh <user@remote> "cat >> .ssh/authorized_keys"
|
~/.ssh/config file format:
Host <my_alias>
HostName <ip>
User <remote username>
To connect through proxy, do
ProxyCommand ssh yurif@csm nc %h %p
or
ProxyJump pi
To execute a remote command on connect (e.g., `cd` to some directory)
RequestTTY yes # or 'force'
RemoteCommand cd /home/katya/yuri; exec /bin/bash
To avoid / reduce connection drops (especially sftp in Nautilus), on the client side, add:
ServerAliveInterval 180
ServerAliveCountMax 2
To apply to all connections add this under Host *, like this:
Host *
ServerAliveInterval 180
ServerAliveCountMax 2.ssh/config file permissions need to be 644.
ssh connection parameters can be configured in /etc/ssh/sshd_config (globally) and in the $HOME/.ssh/config (per host). In particular, look at {Server,Client}Alive intervals which control how a connection is maintained alive, see above.
Taken from here.
sudo vim /etc/hostname
sudo vim /etc/hosts
sudo rebootThe nmap utility scans for open ports, e.g.
sudo nmap -p 22 192.168.1.0/24 scans for machines listening on port 22 (ssh).
.desktop files located at /usr/share/applications. Can link also to remote folders through sftp.
To create a desktop shortuct, need to create a .desktop file at ~/Desktop.
rsync --partial --progress myfile username@server:/path/to/destination
Transfer file. --partial flag allows to resume an interrupted transfer. --remove-source-files and --delete deletes source after transfer. After --remove-source-files directory tree is left.
| Flag | Effect |
|---|---|
--partial |
Allows to resume an interrupted transfer |
-rP |
Shorthand for --recursive --partial --progress
|
-L |
Copy symlinks |
--remove-source-files |
Remove source can be completed using find <dir> -type d -empty -delete
|
--delete |
scp
TODO: Description
| Command | FUnction |
|---|---|
| Ctrl + z | Puts current process in the background |
jobs |
Lists current session jobs. |
fg <id> |
Brings process <id> (from jobs output) to foreground. |
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head |
Processes info |
| Program | Description |
|---|---|
fail2ban |
Service to block repeated accesses to computer/server. |
imagemagick |
Set of cli tools for manipulation of images. In particular, convert converts formats and transforms, display opens image in a display window. |
diffuse |
Graphical comparison allowing manual alignment |
pdftk |
Set of cli tools for dealing with pdfs |
xclip |
Allows to redirect programs output to clipboard |
jq, yq
|
command-line json / yaml parsing respectively |
curl, wget
|
download files. Syntax: wget url -O filename, curl url -o filename. Curl writes to stdout by default. |
tmux, screen
|
Terminal multiplexers. Allow to run programs persistently without being connected. tmux cheatsheet, screen cheatsheet |
htop, glances
|
Display system resources / status. glances can be installed as a python package. Glances reference. |
finger |
command-line utility showing users currently connected to the host |
iotop |
Display I/O system resources (glances does this as well). iotop -oPa
|
timeshift |
Does automatic system backups. Can be configured via shell through /etc/timeshift.json. |
eog |
"Eye of GNOME", displays images. |
shuf |
Shuffles lines of input. Alternatively, can use perl: cat file1 file2 | perl -MList::Util=shuffle -wne 'print shuffle <>;', taken from here. |
lazygit |
Command-line git gui. See git cheatsheet for keybindings. |
nl |
Adds numbering to input lines |
resh |
Enhanced reverse-search the command line link |
ldd |
See which binaries an executable is loading. (windows version: "depends.exe"). lddtree shows them recursively. |
| Command | Action |
|---|---|
unzip file.zip -d destination_folder |
|
unzip -t test.zip |
test if an archive is valid |
unzip -l file.zip |
list contents |
unzip file.zip -x a_particular_file |
exclude file |
zip -r file.zip list of files and directories |
Compresses recursively. |
tar -xvf |
Extracts (verbose) -C or --directory destination dir |
tar -czvf file.tar.gz file-or-dir [-T list-of-files.txt] |
Compresses (-c) using gzip to file (-f). Recurses into directories by default (prevent with --no-recursion). -h Follows symlinks. |
SIZE=$(du -sk RADIal/ | cut -f 1); tar chf - RADIal/ | pv -p -s ${SIZE}k | gzip > $FS_AID_HOME/RADIal.tar.gz |
Compress with progress bar (taken from here) |
7z x file |
Extracts. Need to sudo apt-get install p7zip-full to get 7z. |
| Shortcut | Function |
|---|---|
| c, C | Show file count, sort by file count |
| s | Sort by size |
| pdftk myfile.pdf cat 1-3 5 7-end output outfile.pdf | Output portions of pdf |
Convert PDF into (true) black-and-white (good for sending as fax):
gs -sDEVICE=bmpgray -dNOPAUSE -dBATCH -r300x300 -sOutputFile=./Payement-confirmation.pdf ./Payement-confirmation.pdf
gs -o ./Payement-confirmation-bw.pdf -sDEVICE=pdfwrite -c "/osetrgbcolor {/setrgbcolor} bind def /setrgbcolor {pop [0 0 0] osetrgbcolor} def" -f ./Payement-confirmation-b\&w.pdfConvert PDF to png:
convert -density 700 input.pdf -quality 90 output.pngcurl -sSL
'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xBBEBDCB318AD50EC6865090613B00F1FD2C19886'
| sudo apt-key add -
Documentation here.
${parameter#pattern} |
Deletes shortest (## for longest) match of pattern from beginning of parameter |
${parameter%pattern} |
Deletes shortest (%% for longest) match of pattern from end of parameter |
${parameter/pattern/string} |
Replaces the first match of pattern with string. // for all matches, # and % for matches at beginning and end respectively |
${!variable} |
Refer to the variable who's name is stored in variable (indirection) |
Crons are stored at /var/spool/cron/crontabs.
[sudo] crontab -l |
lists crons. sudo lists root's crons. |
[sudo] crontab -e |
edit. sudo edits root's crons. |
@reboot command |
Run command on startup. Probably should add as root. |
01 04 1 1 1 <command path> |
The five time-and-date fields are as follows: minute (0-59), hour (0-23, 0 = midnight), day (1-31), month (1-12), weekday (0-6, 0 = Sunday). Further line format options and docs. |
| Further documentation |
To schedule a command to run once can use the at command (might need to apt-get install it). For example
echo 'mycommand' | at 16:00 # at now + 3 hoursor
at 16:00 -f some_script.shProblem: AltGr special keys not function Ubuntu 18.04. Solution: Keyboard shortcuts ==> Restore keyboard shortcuts (Super+Escape default).
prefix = right alt
| Key | Shortcut |
|---|---|
| é | prefix+', e |
| è | prefix+`, e |
| á, à | idem. |
| ç, Ç | prefix+, (hold prefix) c / shift+c |
| ï | " + i |
| ĉ | ^+c |
Also see here
Example
conda-workon() {
module load conda; conda activate "$1"
}
_conda-workon_completions()
{
if [ "${#COMP_WORDS[@]}" != "2" ]; then
return
fi
envs_options="`ls $HOME/.conda/envs`"
COMPREPLY=($(compgen -W "${envs_options}" "${COMP_WORDS[1]}"))
}
complete -F '_conda-workon_completions' 'conda-workon'
To enable for all users, add the completion script into /etc/bash_completion.d/
sudo apt install libopencv-dev
sudo apt install libpcl-dev
sudo apt-get install mc
Can choose Lynx-like motion in Options (F9) -> Panel options,
mc -x |
start with mouse |
| Ctrl+o | Back to shell |
| Tab | Switch pane |
| Shift+F6 | Renane file in place |
%d/%f, %D/%F
|
Refers to left dir/file and right dir/file respectively |
| Alt+I |
cd the other pane to the same directory as the current pane. |
| Alt+A | Copy the cwd of the current pane to the command line. |
| [Alt, Ctrl]+s | [Incremental, Forward] search |
| Alt+? | Find file dialog |
| Ctrl+R | Refresh file list in pane. |
| Esc+Tab | Bash auto-completion. |
| F9 ==> Right ==> sftp link | ssh connection. Can use ~/ssh/config hostnames |
| F9 ==> Options ==> Panel ==> Lynx-like motion | Left/right arrow keys exit / enter directories. To naje default set navigate_with_arrows=true in $HOME/.config/mc/ini. |
More information about macros etc. here
And here
And another excellent cheatsheet here
A separate access permissions mechanism for file systems via access lists, complementary to permissions flags.
Docs here: https://linux.die.net/man/1/setfacl
chmod g+rwxs
setfacl -m d:g::rwx
nvidia-smi |
List gpu usage and processes. Good to combine with watch -n for continuous status. |
nvidia-smi -L |
List gpu devices. Other useful flags. |
nvitop / nvtop
|
Top command for nvidia gpus. Can install using python pip |
sudo fuser -v /dev/nvidia* |
List all processes taking up the gpu. Without sudo will list just processes for the current user. |
gsettings set org.gnome.shell.extensions.ding show-trash true
gsettings set org.gnome.shell.extensions.dash-to-dock show-trash false