bash - ghdrako/doc_snipets GitHub Wiki

Fix command

shell built-in fc (fix command) will take you to a default text editor configured in the shell with the last command as the text. Edit it and save it in order to execute it again.

Modifing bash

umask  # change default permissions used when creating a file
alias 
shopt
shopt -s cdspell # enable Bash to correct minormisspelling int its default directories when use cd command
shopt -s checkjobs # list any stopped job that remains when you clse the shell

Bash header

#!/usr/bin/env bash

Reload shell after change in configuration

exec -l $SHELL

..usually as an alternative to:

source ~/.profile

running the $SHELL -l command would "apply" those changes. Running source ~/.profile would do so similarly without starting a new shell, except any variables or functions in your config that are not overwritten will remain set as they were initially.

Variables

Case Sensitivity: Variable names in shell scripts are case-sensitive. This means that VAR and var are considered two distinct variables.

Variables in shell scripting can be categorized into two main types:

shell (local) variables and
environment variables.

Shell (Local) Variables: These variables are defined within the context of a shell and are only accessible within the shell session where they were created. They remain in existence for the duration of the shell session or until explicitly unset.

Environment Variables: These variables are inherited by any child processes or subshells spawned from the original shell session. They provide a means to influence the behavior of these child processes and stand as a means for communication between the different processes.

To promote a local variable to an environment variable, the export command is used.

export variable_name

The variable name would now be accessible in any child processes of the current shell session.

Multiline variables

To correctly handle multi-line strings, use a combination of single quotes for literal strings or double quotes with embedded variables and escape sequences

#!/bin/bash

# Define multi-line text
multi_line_text="
This is line one.
This is line two, and this is line three.
"

# Echo the multi-line text
echo "$multi_line_text"

Built-in variables:

Parameter Name	Usage	Description
0	"$0"	Contains the name, or the path, of the script. This is not always reliable.
1 2 etc.	"$1" etc.	Positional Parameters contain the arguments that were passed to the current script or function.
*	"$*"	Expands to all the words of all the positional parameters. Double quoted, it expands to a single string containing them all, separated by the first character of the IFS variable (discussed later).
@	"$@"	Expands to all the words of all the positional parameters. Double quoted, it expands to a list of them all as individual words.
#	$#	Expands to the number of positional parameters that are currently set.
?	$?	Expands to the exit code of the most recently completed foreground command.
$	$	Expands to the PID (process ID number) of the current shell.
!	$!	Expands to the PID of the command most recently executed in the background.
_	"$_"	Expands to the last argument of the last command that was executed.

echo "Last program's return value: $?"
echo "Script's PID: $$"
echo "Number of arguments passed to script: $#"
echo "All arguments passed to script: $@"
echo "Script's arguments separated into different variables: $1 $2..."
echo "I'm in $(pwd)" # execs `pwd` and interpolates output
echo "I'm in $PWD" # interpolates the variable

Variables that the shell provides for you:

BASH_VERSION: Contains a string describing the version of Bash.
HOSTNAME: Contains the hostname of your computer, I swear. Either short or long form, depending on how your computer is set up.
PPID: Contains the PID of the parent process of this shell.
PWD: Contains the current working directory.
RANDOM: Each time you expand this variable, a (pseudo)random number between 0 and 32767 is generated.
UID: The ID number of the current user. Not reliable for security/authentication purposes, alas.
COLUMNS: The number of characters that fit on one line in your terminal. (The width of your terminal in characters.)
LINES: The number of lines that fit in your terminal. (The height of your terminal in characters.)
HOME: The current user's home directory.
PATH: A colon-separated list of paths that will be searched to find a command, if it is not an alias, function, builtin command, or shell keyword, and no pathname is specified.
PS1: Contains a string that describes the format of your shell prompt.
TMPDIR: Contains the directory that is used to store temporary files (by the shell).

RANDOM

https://tldp.org/LDP/abs/html/randomvar.html

# $RANDOM returns a different random integer at each invocation.
# Nominal range: 0 - 32767 (signed 16-bit integer).


RANGE=500
echo
number=$RANDOM
let "number %= $RANGE"
#           ^^
echo "Random number less than $RANGE  ---  $number"
echo


FLOOR=200
number=0   #initialize
while [ "$number" -le $FLOOR ]
do
  number=$RANDOM
done
echo "Random number greater than $FLOOR ---  $number"
echo

# Combine above two techniques to retrieve random number between two limits.
number=0   #initialize
while [ "$number" -le $FLOOR ]
do
  number=$RANDOM
  let "number %= $RANGE"  # Scales $number down within $RANGE.
done
echo "Random number between $FLOOR and $RANGE ---  $number"
echo

# Loop, and randomly pick elements out of the list
range="${#mylist[@]}"
for ((i=0; i<35; i++)); do
  random_element="$(( $RANDOM % $range ))"
  echo "${mylist[$random_element]}" >> $WORD_FILE
done

Variable Types

Array: declare -a variable: The variable is an array of strings.
Associative array: declare -A variable: The variable is an associative array of strings (bash 4.0 or higher).
Integer: declare -i variable: The variable holds an integer. Assigning values to this variable automatically triggers Arithmetic Evaluation.
Read Only: declare -r variable: The variable can no longer be modified or unset.
Export: declare -x variable: The variable is marked for export which means it will be inherited by any child process.

Sourcing

When the script that you ran (or any other program, for that matter) finishes executing, its environment is discarded. The environment of the first script will be same as it was before the second script was called, although of course some of bash's special parameters may have changed (such as the value of $?, the return value of the most recent command).

A child process can never affect any part of the parent's environment, which includes its variables, its current working directory, its open files, its resource limits, etc.

What you can do is to source the script, instead of running it as a child. You can do this using the . (dot) command:

. ./myscript   #runs the commands from the file myscript in this environment

This is often called dotting in a script. The . tells BASH to read the commands in ./myscript and run them in the current shell environment. Since the commands are run in the current shell, they can change the current shell's variables, working directory, open file descriptors, functions, etc.

Note that Bash has a second name for this command, source, but since this works identically to the . command, it's probably easier to just forget about it and use the . command, as that will work everywhere.

If the command you execute is a function, not a script, it will be executed in the current shell. Therefore, it's possible to define a function to do what we tried to do with an external file in the examples above, without needing to "dot in" or "source" anything. Define the following function and then call it simply by typing mycd:

   mycd() { cd /tmp; }

Put it in ~/.bashrc or similar if you want the function to be available automatically in every new shell you open.

Inject .env into your bash session/environment

export $(cat .env | xargs)

Reading a value from input:

echo "What's your name?"
read Name # Note that we didn't need to declare a new variable
echo Hello, $Name!
date | tee file1 file2
uptime | tee -a file2

read -p "Are you sure you want to merge 'develop' into 'staging'? (y/N)" -n 1 -r
echo # we like having a new line

if [ $REPLY =~ ^[Yy]$ ](/ghdrako/doc_snipets/wiki/-$REPLY-=~-^[Yy]$-)
then
  git merge develop --ff-only
  git push
fi

Default value

A=${A:-hello} # if A is undefined or empty set hello

FOO="${VARIABLE:-default}"  # If variable not set or null, use default.
# If VARIABLE was unset or null, it still is after this (no assignment done).

FOO="${VARIABLE:=default}"  # If variable not set or null, set it to default.

Redirection

exec > filename # redirects stdout to the designated file
command &> filename # redirects both the stdout and the stderr of command to filename
command > file 2>&1 # sa above

ls > dirlist 2>&1   # direct both standard output and standard error to the file dirlist 
ls 2>&1 > dirlist  # will only direct standard output to dirlist (useful option for programmers )
date | tee file1 file2
uptime | tee -a file2
exec &> >(tee -a "$log_file")
exec &> >(tee -a "$log_file")

Glob patterns

A glob pattern is a search query that contains wildcard characters. Bash allows three wildcard characters: *, ? and ````[. The asterisk stands for any number of any characters. A question mark means a single character of any kind. Square brackets indicate a set of characters at a specific position. For example, the pattern [cb]at.txt matches the cat.txt and bat.txt

Text processing

Contents=$(cat file.txt)

# prints last 10 lines of file.txt
tail -n 10 file.txt

# prints first 10 lines of file.txt
head -n 10 file.txt

# sort file.txt's lines
sort file.txt

# report or omit repeated lines, with -d it reports them
uniq -d file.txt

# prints only the first column before the ',' character
cut -d ',' -f 1 file.txt

# replaces every occurrence of 'okay' with 'great' in file.txt
# (regex compatible)
sed -i 's/okay/great/g' file.txt

# print to stdout all lines of file.txt which match some regex
# The example prints lines which begin with "foo" and end in "bar"
grep "^foo.*bar$" file.txt

# pass the option "-c" to instead print the number of lines matching the regex
grep -c "^foo.*bar$" file.txt

# Other useful options are:
grep -r "^foo.*bar$" someDir/ # recursively `grep`
grep -n "^foo.*bar$" file.txt # give line numbers
grep -rI "^foo.*bar$" someDir/ # recursively `grep`, but ignore binary files

# perform the same initial search, but filter out the lines containing "baz"
grep "^foo.*bar$" file.txt | grep -v "baz"

# if you literally want to search for the string,
# and not the regex, use fgrep (or grep -F)
fgrep "foobar" file.txt

Condition

# succeed with the cd or bail with a message
cd $DIR || { echo "cd to $DIR failed." ;  exit ; }
# the && and || operators are of equal precedence and are left Associated
[ -n "$DIR" ] && [ -d "$DIR" ] && cd "$DIR" || exit 2

IF structure

[ is a Bash builtin, while [[ is a Bash keyword

[...]

if [ $Name != $USER ]
then
    echo "Your name isn't your username"
else
    echo "Your name is your username"
fi
# True if the value of $Name is not equal to the current user's login username

# NOTE: if $Name is empty, bash sees the above condition as:
if [ != $USER ]
# which is invalid syntax
# so the "safe" way to use potentially empty variables in bash is:
if [ "$Name" != $USER ] ...
# which, when $Name is empty, is seen by bash as:
if [ "" != $USER ] ...
# which works as expected

# There is also conditional execution
echo "Always executed" || echo "Only executed if first command fails"
# => Always executed
echo "Always executed" && echo "Only executed if first command does NOT fail"
# => Always executed
# => Only executed if first command does NOT fail


# To use && and || with if statements, you need multiple pairs of square brackets:
if [ "$Name" == "Steve" ] && [ "$Age" -eq 15 ]
then
    echo "This will run if $Name is Steve AND $Age is 15."
fi

if [ "$Name" == "Daniya" ] || [ "$Name" == "Zach" ]
then
    echo "This will run if $Name is Daniya OR Zach."
fi

#### [ ... ](/ghdrako/doc_snipets/wiki/-...-)
`[ ... ](/ghdrako/doc_snipets/wiki/-...-)` is more robust than `[ ... ]` because it handles special characters
and logical operators more gracefully. It avoids issues like word
splitting and allows for advanced features such as regex-style pattern
matching (==, !=). Unlike [ ... ], it does not require special quoting
for variables containing spaces.

`[ ... ](/ghdrako/doc_snipets/wiki/-...-)` is specific to Bash and not available in all shells. In POSIX-
compliant scripts, `[ ... ]` should be used.Inside `[ ... ](/ghdrako/doc_snipets/wiki/-...-)`, pattern matching
works without needing * to be quoted, whereas `[ ... ]` requires careful
quoting to avoid globbing and expansion.


# There is also the `=~` operator, which tests a string against a Regex pattern:
[email protected]
if [ "$Email" =~ [a-z]+@[a-z]{2,}\.(com](/ghdrako/doc_snipets/wiki/net|org)-)
then
    echo "Valid email!"
fi
# Note that =~ only works within double [ ](/ghdrako/doc_snipets/wiki/-) square brackets,
# which are subtly different from single [ ].
# See http://www.gnu.org/software/bash/manual/bashref.html#Conditional-Constructs for more on this.

# Numeric Conditions
[ NUM -eq NUM ](/ghdrako/doc_snipets/wiki/-NUM--eq-NUM-) Equal
[ NUM -ne NUM ](/ghdrako/doc_snipets/wiki/-NUM--ne-NUM-) Not equal
[ NUM -lt NUM ](/ghdrako/doc_snipets/wiki/-NUM--lt-NUM-) Less than
[ NUM -le NUM ](/ghdrako/doc_snipets/wiki/-NUM--le-NUM-) Less than or equal to
[ NUM -gt NUM ](/ghdrako/doc_snipets/wiki/-NUM--gt-NUM-) [ NUM -ge NUM ](/ghdrako/doc_snipets/wiki/-NUM--ge-NUM-) Greater than or equal to

# String Conditions
[ STRING == STRING ](/ghdrako/doc_snipets/wiki/-STRING-==-STRING-) # Equal
[ STRING != STRING ](/ghdrako/doc_snipets/wiki/-STRING-!=-STRING-) # Not Equal
[ -z STRING ](/ghdrako/doc_snipets/wiki/--z-STRING-)        # Empty string
[ -n STRING ](/ghdrako/doc_snipets/wiki/--n-STRING-)        # Not empty string
[ STRING =~ STRING ](/ghdrako/doc_snipets/wiki/-STRING-=~-STRING-) # Regular expression match

# check string pattern matching
input="hello123"
if [ $input == hello* ](/ghdrako/doc_snipets/wiki/-$input-==-hello*-); then
echo "The input starts with 'hello'"
else
echo "The input does not start with 'hello'"
fi

number=10
if [ $number -gt 5 && $number -lt 20 ](/ghdrako/doc_snipets/wiki/-$number--gt-5-&&-$number--lt-20-); then
echo "The number is between 5 and 20"
fi

# Is the length of the variable zero? i.e., empty or null
if [ -z "$VAR" ](/ghdrako/doc_snipets/wiki/--z-"$VAR"-); then
echo empty
fi

# This checks for a nonzero length, i.e., not empty, not null
if [ -n "$VAR" ](/ghdrako/doc_snipets/wiki/--n-"$VAR"-); then
echo "VAR has a value:" $VAR
fi
# Same here
if [ "$VAR" ](/ghdrako/doc_snipets/wiki/-"$VAR"-); then
echo even easier this way
fi

#File Conditions
[ -f FILE ](/ghdrako/doc_snipets/wiki/--f-FILE-) # Is a file
[ -d FILE ](/ghdrako/doc_snipets/wiki/--d-FILE-) # Is a directory
[ -e FILE ](/ghdrako/doc_snipets/wiki/--e-FILE-) # Exists
[ -r -w -x FILE ](/ghdrako/doc_snipets/wiki/--r--w--x-FILE-) # Is readable, Writable, executable
[ -h FILE ](/ghdrako/doc_snipets/wiki/--h-FILE-) # Is symbolic link

#File Conditions Example
if [ ! -f ./pdfgen/pdfgen ]; then
    echo "Building pdfgen binary"
    npm run --prefix pdfgen build:linux
else
    echo "Pdfgen binary already exists, skipping build"
fi


if [ ! -L /usr/local/bin/heroku ];
then
    wget https://cli-assets.heroku.com/branches/stable/heroku-linux-amd64.tar.gz
    sudo mkdir -p /usr/local/lib /usr/local/bin
    sudo tar -xvzf heroku-linux-amd64.tar.gz -C /usr/local/lib
    sudo ln -s /usr/local/lib/heroku/bin/heroku /usr/local/bin/heroku
fi

# Boolean conditions:
a|[ ! EXPR ](/ghdrako/doc_snipets/wiki/-!-EXPR-) |Not
a|[ BOOL && BOOL ](/ghdrako/doc_snipets/wiki/-BOOL-&&-BOOL-) |And
a|[ BOOL ](/ghdrako/doc_snipets/wiki/|-BOOL-) |OR

the double bracket syntax supports an additional comparison operator, =~, which allows the use of regular expressions.

if [ $FN =~ .*xyzz*.*jpg ](/ghdrako/doc_snipets/wiki/-$FN-=~-.*xyzz*.*jpg-); then ...

Check if an environment variable is set in bash

# long
if [ -z "${CIRCLE_BRANCH}" ](/ghdrako/doc_snipets/wiki/--z-"${CIRCLE_BRANCH}"-); then
    npm run redis-cli flushall
fi
 
npm run sync

# one-liner
[-z "${CIRCLE_BRANCH}"] && npm run redis-cli flushall; npm run sync

Aliases

alias ping='ping -c 5' # Redefine command `ping` as alias to send only 5 packets

\ping 192.168.1.1  # Escape the alias and use command with this name instead

alias -p # Print all aliases

An alias will take precedence over external programs and even built-in commands. That means we can also overshadow standard utilities like cd or ls and replace them with something else.

WHILE loop

while [ true ]
do
    echo "loop body here..."
    break
done

TRAP

The trap command allows you to execute a command whenever your script receives a signal.

trap "rm $TEMP_FILE; exit" SIGHUP SIGINT SIGTERM # execute `rm` if it receives any of the 
# three listed signals.

MAN HELP INFO

# Read Bash shell built-ins documentation with the bash `help` built-in:
help
help help
help for
help return
help source
help .

# Read Bash manpage documentation with `man`
apropos bash
man 1 bash
man bash

# Read info documentation with `info` (`?` for help)
apropos info | grep '^info.*('
man info
info info
info 5 info

# Read bash info documentation:
info bash
info bash 'Bash Features'
info bash 6
info --apropos bash

Create directory structure

mkdir -p modules/gcs-static-website-bucket

Repeat command using !

history
!<number associated with the command>
!1  # rerun first command in history
!!  # rerun the most recent command

This comes in handy if you execute a command that should have been run with sudo:

$ mkdir /var/website
mkdir: /var/website: Permission denied
$ sudo !!

Switching Between Directories

pushd - changes your current working directory and adds it to a directory stack, or a list of directories.
popd - removes the first entry from the stack, and then changes your location to the new first entry:

$ dirs -v

0  /usr/bin
1  /tmp
2  /var
3  ~

pushd +2 #  Jump to the /var folder and moves the /var folder to the top of the stack.
cd ~3    #  switch to an entry in the stack, but this does modify the stack

PIPE SYMBOL AND MULTIPLE COMMANDS

(ls | sort; echo "the contents of /tmp:"; ls /tmp) > myfile1

each of the preceding commands inside the parentheses spawns a subshell (which is an extra process that consumes memory and CPU). You can avoid spawning subshells by using {} instead of ()

{ ls | sort; echo "the contents of /tmp:"; ls /tmp } > myfile1

Suppose that you want to set a variable, execute a command, and invoke a second command via a pipe

(name=SMITH cmd1) | cmd2

Use the double ampersand && symbol if you want to execute a command only if a prior command succeeds.

mkdir -p /tmp/abc/def && cd /tmp/abc && ls -l