Things I forget - fordsfords/fordsfords.github.io GitHub Wiki

Things I forget. This is the sequel to http://www.geeky-boy.com/w/Sford_cheat_sheets.html

Table of Contents VIM Regular Expressions Unux stuff Troubleshooting tools Shared ssh keys Yum Stuff Shell Stuff Docker Stuff Language Stuff Perl Stuff C Stuff GDB Python Stuff PDB POSIX Sockets Increase max socket buffer size SO_RCVTIMEO vs O_NONBLOCK Chicken of the VNC Hardware Licensing Questions Npcap License and Wireshark iptables Simulate Net Delay Cute Shell Tricks noclobber process substitution See Process Environment Sed Version Rotate File Lines Noglob

Vim Tips

• https://regex101.com - RE tester/debugger
• https://perldoc.perl.org/perlrecharclass - Perl character classes

Being an old guy, I tend to use tools like "netstat", "ifconfig", etc. But there has been a migration (at least on Linux, which is increasingly becoming the only relevant Unix) to a new toolset.

Replacement for:

• netstat is ss. See https://www.redhat.com/sysadmin/ss-command
• netstat -r is ip route.
• netstat -i is ip -s link.
• netstat -g is ip maddr.
• ifconfig is ip addr. See https://www.redhat.com/sysadmin/ifconfig-vs-ip
• arp -v is ip -s neigh.
• traceroute is tracepath

Enter "ip help" to get list of objects, then "ip OBJECT help" to get detailed help.

Other network tools:

• tcpdump
• wireshark
• ethtool
• ping
• traceroute/tracepath
• route
• lsof

General tools:

• strace - system calls.
• ltrace - all library calls (not just system).
• gstack (a.k.a. pstack) - stack trace of a running process (actually a shell script that invokes gdb).
• gcore - get a core fore file of a running process, but lets the program continue running (shell script that invokes gdb).

Let's say your account name is "sford" and you want to be able to log into the machine "blunjo" from your laptop without password.

Log into blunjo. These commands might not be necessary, but for completeness:

cd $HOME
mkdir .ssh
chmod 700 .ssh

Then for sure do these (still on blunjo):

cd .ssh
touch authorized_keys
chmod 600 *

From your laptop, these commands are almost certainly not necessary, but for completeness:

cd $HOME
mkdir .ssh
chmod 700 .ssh

Then for sure do:

cd .ssh
ls

If the files "id_rsa" (private key) and "id_rsa.pub" (public key) already exist, skip the following step. Otherwise enter:

ssh-keygen -b 2048 -t rsa -C sford

Just hit enter for the prompts.

Then (still on laptop):

scp id_rsa.pub sford@blunjo:.ssh/authorized_keys

You'll need to supply your password this time. But from now on, it should work without password.

Find the yum package that contains the Java compiler:

yum provides '*/javac'

POSIX: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html

• Functions

usage() {
  echo $1
}

• Background process PID:

blah &
BLAH_PID=$!

• Getopts:

while getopts "ht:" OPTION; do :
  case $OPTION in
    h) help ;;
    t) echo $OPTARG ;;
    \?) usage ;;
  esac
done
shift `expr $OPTIND - 1`  # Remove options leaving $1 as the first positional parameter

• Catch interrupt/failures

trap "kill $BLAH_PID; exit 1" 1 2 3 15

See also Docker and Docker Hub Notes.

• Documentation: https://docs.docker.com/engine/reference/commandline/docker//
• Log into running container (the "-it" makes it interactive):

sudo docker ps
sudo docker exec -u sford -it (container) /bin/bash

• Free up disk space by killing unused stuff:

docker system prune

• Various commands:

docker images -a            # list all images
docker container ls -a      # list all containers
docker ps -a                # Same thing? More options?
docker rm (container)       # Either hex ID or name
docker rmi (image)          # Either hex ID or name
docker commit (container) (image)  # Create new image from container
docker cp {hostfile|-} container:containerfile      # Careful: sym links copied as sym links; see -L
docker cp container:containerfile {hostfile|-}
docker save (image) >image.tar
docker load <image.tar
docker login                # log into docker hub
docker push/pull...         # docker hub repo stuff

See: http://www.geeky-boy.com/rstone/

See: https://github.com/fordsfords/skeleton

LANG=C perl -anle 'print $_;'

• -a = auto-split into @F
• -n = automatic "while(<>)" loop
• -l = auto-chomp, auto print \n
• -e program

uint64_t

• literal: uint64_t i = UINT64_C(1000000000)
• sscanf(str, "%" SCNu64 ", &i);
• printf("%"PRIu64"\n", i);

See: gdb notes

See: https://darkdust.net/files/GDB%20Cheat%20Sheet.pdf

• To shell out:

import os
os.system("sh")

https://pubs.opengroup.org/onlinepubs/9699919799/

# sysctl -w net.core.rmem_max=128000000
# vi /etc/sysctl.conf
net.core.rmem_max=128000000

I've seen complaints that SO_RCVTIMEO is not consistently and/or reliably implemented everywhere. For example, from POSIX:

"If option_name is equal to SO_RCVTIMEO or SO_SNDTIMEO and the implementation supports setting the option, it is unspecified whether the struct timeval pointed to by option_value is stored as provided by this function or is rounded up to align with the resolution of the clock being used."

(emphasis mine)

To emulate middle button:

1. Pull down "Connection" -> "Connection profiles..."
2. Select "Mouse" tab
3. Center Mouse Button: Modifier Click

(Thanks Jerin!)

• See brand of NICs:

lspci | grep -i ethernet

• Basic CPU/NUMA info:

lscpu

When installing Wireshark, you have the option of also installing npcap. It gives a licensing blurb which, as of 31-Dec-2022 says it is NOT openSource, and potentially requires purchase of a license. "The standard (free) version is usually limited to installation on five systems." But keep reading:

Copies of Npcap do not count toward the five copy, five computer, or five user limitations imposed by this section if they are installed and used solely in conjunction with any of the following software:

• The Nmap Security Scanner, as distributed from https://nmap.org
• The Wireshark network protocol analyzer, as distributed from https://www.wireshark.org/
• Microsoft Defender for Identity, as distributed from https://www.microsoft.com/en-us/microsoft-365/security/identity-defender

So Wireshark users, even corporate users, are OK to install npcap with Wireshark.

iptables -L  # list current tables.
iptables -F  # delete all rules.
# Block port 12000 (in and out)
iptables -A INPUT -p tcp --dport 12000 -j DROP; iptables -A OUTPUT -p tcp --sport 12000 -j DROP

I haven't actually played with these much.

sudo tc qdisc add dev vlan448 root netem delay 30ms

sudo tc qdisc delete dev vlan448 root netem delay 30ms

If you have multiple shell processes on a host and you only want the first one to dump its results:

DO_DUMP=0
set -o noclobber
if date >outfile; then :
  DO_DUMP=1
fi >/dev/null  # we don't need to see noclobber errors

if [ "$DO_DUMP" -eq 1 ]; then :
  dump results
fi >>outfile 2>&1

Compare two directories filenames:

diff <(ls -1 dir1) <(ls -1 dir2)

Process stdout and stderr separately:

process_foo > >(format_foo >foo.txt) 2> >(alert_operator)

Note the space between the two ">" characters - this is needed. Without the space, ">>" is treated as the append redirection.

The /proc/12345/environ file has nulls instead of newlines. I guess that's because an env var can contain newlines, which means the method below is brittle.

$ tr '\0' '\n' </proc/12345/environ>| egrep "^HOME="
HOME=/home/sford

BSD and GNU sed differ in annoying ways. This works for both:

sed -i.bak -e "s/x/y/" x

The lack of space between "-i" and ".bak" is important.

Move the first line to the end of the file. (see blog entry.)

sed -i.bak -e '1h;1d;$G' my_file.txt

EXCL='--exclude *.o' # default
set -o noglob
rsync -a $EXCL my_src_dir/ orion:my_src_dir
set +o noglob