shellsheet - feliyur/exercises GitHub Wiki
Location of config/permissions on Ubuntu |
/etc/sudoers , /etc/groups
|
Show storage devices and UUID | lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,UUID |
Execute monitoring command every n seconds | watch -n <sec> <command> <command args> |
Compute directory size, show filesystem info. |
du , df , ncdu
|
Get / change hostname |
hostname to get, sudo hostnamectl set-hostname to set (or sudo edit /etc/hostname ) |
Get ip address |
ifconfig -a old style. New: ip address or ip a . |
Who is logged on |
who -H , who -u , w , users , ps au the latter shows processes for each user. |
Processes info | ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head |
Copy symlink | cp -R --preserve=links |
Clean up system logs | sudo journalctl --vacuum-size=100M |
Find and delete empty directories | find /path/to/directory -type d -empty -delete |
Find and filter files | find /path/to/directory -type [f,d] -name 'supports wildcards' [-maxdepth <N> -mindepth <M>] |
More complex find usage example with exec |
find /path/to/directory -type d -exec chmod 770 {} \; -type f -exec chmod 660 {} \; . Mind the \; which ends the command, and {} which is placeholder of current item. |
List contents recursively | tree [-L <max depth>] |
Set a timeout for a blocking command |
timeout <n_seconds> <command and arguments> e.g. timeout 100 ssh <host> will close the ssh connection after 100 seconds. |
Format a partition to ext4 | sudo mkfs.ext4 /dev/nvme0n1 |
Set execute permissions only to subdirectories, but not to files | chmod -R rwX <target dir> |
get and control file access lists | getfacl / setfacl |
List groups user is part of |
id -a , groups
|
List path and permissions to target | namei -l <target> |
To search back in shell command history use Ctrl+r (this is called "reverse-i-search"). Can use Ctrl+s to search forward, but possibly need to remap an overshadowing shortcut for this key combination ("stop terminal sequence"), adding stty stop ^J
to .bashrc
(see here).
!<command prefix>
executes last command starting with prefix. history
shows command history.
Safe fstab line:
UUID=<device UUID> <mountpoint> none <options> 0 0
Mountpoint must exist. Can check UUID with lsblk
Worth to almost always use nofail
to allow the system to boot even if the drive cannot be mounted. E.g. replace <options> with defaults,nofail
.
For network-facing systems, worth to use safe options nosuid,nodev
.
Mount ntfs filesystem:
UUID=F4E2C7FAE2C7BF5C /media/Windows ntfs defaults,nofail,nls=utf8,umask=000,dmask=027,fmask=137,uid=1000,gid=1000,windows_names 0 0
Can use lsblk -o NAME,MOUNTPOINT,SIZE,FSTYPE,UUID
command to list drives and found out UUID of an unmounted drive.
Create new user | sudo adduser [username] |
Add user to group | sudo usermod -a -G groupName userName |
To hide a user from login screen see here:
sudo vim /var/lib/AccountsService/users/[username]
Add / edit the following two lines:
[User]
SystemAccount=true
ssh <server> -C '<command>' |
Run command remotely and disconnect. |
ssh <server> -L<local-port>:localhost:<remote-port> |
maps local port to remote |
ssh <server> -R<local-port>:localhost:<remote-port> |
maps remote port to local (reverse tunnel) |
ssh <params> -N |
Do not execute a remote command (useful for port forwarding). |
Running ssh in the background (e.g. port-forwarding).
Installing: sudo apt-get install openssh-server
Generate ssh key |
ssh-keygen -t ed25519 -C "[email protected]" Or ssh-keygen -t rsa -b 4096 -C "[email protected]"
|
Add local ssh key to remote authorized_keys |
ssh-copy-id <username@remote> . On Windows: type <key .pub file> | ssh <user@remote> "cat >> .ssh/authorized_keys"
|
~/.ssh/config
file format:
Host <my_alias>
HostName <ip>
User <remote username>
To connect through proxy, do
ProxyCommand ssh yurif@csm nc %h %p
or
ProxyJump pi
To execute a remote command on connect (e.g., `cd` to some directory)
RequestTTY yes
RemoteCommand cd /home/katya/yuri; exec /bin/bash
To avoid / reduce connection drops (especially sftp in Nautilus), on the client side, add:
ServerAliveInterval 180
ServerAliveCountMax 2
To apply to all connections add this under Host *, like this:
Host *
ServerAliveInterval 180
ServerAliveCountMax 2
.ssh/config
file permissions need to be 644
.
ssh connection parameters can be configured in /etc/ssh/sshd_config (globally) and in the $HOME/.ssh/config (per host). In particular, look at {Server,Client}Alive intervals which control how a connection is maintained alive, see above.
Taken from here.
sudo vim /etc/hostname
sudo vim /etc/hosts
sudo reboot
The nmap
utility scans for open ports, e.g.
sudo nmap -p 22 192.168.1.0/24
scans for machines listening on port 22 (ssh).
.desktop
files located at /usr/share/applications
. Can link also to remote folders through sftp.
To create a desktop shortuct, need to create a .desktop
file at ~/Desktop.
rsync --partial --progress myfile username@server:/path/to/destination
Transfer file. --partial
flag allows to resume an interrupted transfer. --remove-source-files
and --delete
deletes source after transfer. After --remove-source-files
directory tree is left.
Flag | Effect |
---|---|
--partial |
Allows to resume an interrupted transfer |
-rP |
Shorthand for --recursive --partial --progress
|
-L |
Copy symlinks |
--remove-source-files |
Remove source can be completed using find <dir> -type d -empty -delete
|
--delete |
scp
TODO: Description
Ctrl + z | Puts current process in the background |
jobs |
Lists current session jobs. |
fg <id> |
Brings process <id> (from jobs output) to foreground. |
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head |
Processes info |
fail2ban |
Service to block repeated accesses to computer/server. |
imagemagick |
Set of cli tools for manipulation of images. In particular, convert converts formats and transforms, display opens image in a display window. |
diffuse |
Graphical comparison allowing manual alignment |
pdftk |
Set of cli tools for dealing with pdfs |
xclip |
Allows to redirect programs output to clipboard |
jq , yq
|
command-line json / yaml parsing respectively |
curl , wget
|
download files. Syntax: wget url -O filename , curl url -o filename . Curl writes to stdout by default. |
tmux , screen
|
Terminal multiplexers. Allow to run programs persistently without being connected. tmux cheatsheet, screen cheatsheet |
htop , glances
|
Display system resources / status. glances can be installed as a python package. Glances reference. |
finger |
command-line utility showing users currently connected to the host |
iotop |
Display I/O system resources (glances does this as well). iotop -oPa
|
timeshift |
Does automatic system backups. Can be configured via shell through /etc/timeshift.json . |
eog |
"Eye of GNOME", displays images. |
shuf |
Shuffles lines of input. Alternatively, can use perl: cat file1 file2 | perl -MList::Util=shuffle -wne 'print shuffle <>;' , taken from here. |
lazygit |
Command-line git gui. See git cheatsheet for keybindings. |
nl |
Adds numbering to input lines |
unzip file.zip -d destination_folder |
|
unzip -t test.zip |
test if an archive is valid |
unzip -l file.zip |
list contents |
unzip file.zip -x a_particular_file |
exclude file |
zip -r file.zip list of files and directories |
Compresses recursively. |
tar -xvf |
Extracts (verbose) -C or --directory destination dir |
tar -czvf file.tar.gz file-or-dir [-T list-of-files.txt] |
Compresses (-c) using gzip to file (-f). Recurses into directories by default (prevent with --no-recursion ). -h Follows symlinks. |
tar chf - RADIal/ | pv -p -s ${SIZE}k | gzip > $FS_AID_HOME/RADIal.tar.gz |
Compress with progress bar |
7z x file |
Extracts. Need to sudo apt-get install p7zip-full to get 7z. |
c, C | Show file count, sort by file count |
s | Sort by size |
pdftk myfile.pdf cat 1-3 5 7-end output outfile.pdf | Output portions of pdf |
Convert PDF into (true) black-and-white (good for sending as fax):
gs -sDEVICE=bmpgray -dNOPAUSE -dBATCH -r300x300 -sOutputFile=./Payement-confirmation.pdf ./Payement-confirmation.pdf
gs -o ./Payement-confirmation-bw.pdf -sDEVICE=pdfwrite -c "/osetrgbcolor {/setrgbcolor} bind def /setrgbcolor {pop [0 0 0] osetrgbcolor} def" -f ./Payement-confirmation-b\&w.pdf
Convert PDF to png:
convert -density 700 input.pdf -quality 90 output.png
curl -sSL
'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xBBEBDCB318AD50EC6865090613B00F1FD2C19886'
| sudo apt-key add -
Documentation here.
${parameter#pattern} |
Deletes shortest (## for longest) match of pattern from beginning of parameter |
${parameter%pattern} |
Deletes shortest (%% for longest) match of pattern from end of parameter |
${parameter/pattern/string} |
Replaces the first match of pattern with string . // for all matches, # and % for matches at beginning and end respectively |
${!variable} |
Refer to the variable who's name is stored in variable (indirection) |
Crons are stored at /var/spool/cron/crontabs
.
[sudo] crontab -l |
lists crons. sudo lists root's crons. |
[sudo] crontab -e |
edit. sudo edits root's crons. |
@reboot command |
Run command on startup. Probably should add as root. |
01 04 1 1 1 <command path> |
The five time-and-date fields are as follows: minute (0-59), hour (0-23, 0 = midnight), day (1-31), month (1-12), weekday (0-6, 0 = Sunday). Further line format options and docs. |
Further documentation |
To schedule a command to run once can use the at
command (might need to apt-get install
it). For example
echo 'mycommand' | at 16:00 # at now + 3 hours
or
at 16:00 -f some_script.sh
Problem: AltGr special keys not function Ubuntu 18.04. Solution: Keyboard shortcuts ==> Restore keyboard shortcuts (Super+Escape default).
prefix = right alt
Key | Shortcut |
---|---|
é | prefix+', e |
è | prefix+`, e |
á, à | idem. |
ç, Ç | prefix+, (hold prefix) c / shift+c |
ï | " + i |
ĉ | ^+c |
Also see here
Example
conda-workon() {
module load conda; conda activate "$1"
}
_conda-workon_completions()
{
if [ "${#COMP_WORDS[@]}" != "2" ]; then
return
fi
envs_options="`ls $HOME/.conda/envs`"
COMPREPLY=($(compgen -W "${envs_options}" "${COMP_WORDS[1]}"))
}
complete -F '_conda-workon_completions' 'conda-workon'
To enable for all users, add the completion script into /etc/bash_completion.d/
sudo apt install libopencv-dev
sudo apt install libpcl-dev
sudo apt-get install mc
Can choose Lynx-like motion in Options (F9) -> Panel options,
mc -x |
start with mouse |
Ctrl+o | Back to shell |
Tab | Switch pane |
Shift+F6 | Renane file in place |
%d/%f , %D/%F
|
Refers to left dir/file and right dir/file respectively |
Alt+I |
cd the other pane to the same directory as the current pane. |
Alt+A | Copy the cwd of the current pane to the command line. |
Ctrl+R | Refresh file list in pane. |
Esc+Tab | Bash auto-completion. |
F9 ==> Right ==> sftp link | ssh connection. Can use ~/ssh/config hostnames |
F9 ==> Options ==> Panel ==> Lynx-like motion | Left/right arrow keys exit / enter directories. To naje default set navigate_with_arrows=true in $HOME/.config/mc/ini . |
More information about macros etc. here
And here
A separate access permissions mechanism for file systems via access lists, complementary to permissions flags.
Docs here: https://linux.die.net/man/1/setfacl
chmod g+rwxs
setfacl -m d:g::rwx
nvidia-smi |
List gpu usage and processes. Good to combine with watch -n for continuous status. |
nvidia-smi -L |
List gpu devices. Other useful flags. |
nvitop / nvtop
|
Top command for nvidia gpus. Can install using python pip |
sudo fuser -v /dev/nvidia* |
List all processes taking up the gpu. Without sudo will list just processes for the current user. |
gsettings set org.gnome.shell.extensions.ding show-trash true
gsettings set org.gnome.shell.extensions.dash-to-dock show-trash false