Tips in Linux commands and shell

Dealing with variables

Parameter expansion – assign a default value to variable

Let’s say we have this script (filename: demo.sh) which will read $1 as a variable, however, sometimes the users may run the command without $1.

user_input=$1
echo "[Hello $user_input]"

When users just enter $ ./demo.sh , the output will be just like [Hello ]

Now, let’s update the script to :

user_input="${1:-World}"
echo "[Hello $user_input]"

Even users just run it in : $ ./demo.sh, the output will be : [Hello World], and if users run it like : $ ./demo.sh All, then the output will be : [Hello All].

Extract the string from variable

Check the examples :

$ word="Hello World"

$ echo ${word:0:1}
H

$ echo ${word:0:2}
He

$ echo ${word:0:3}
Hel

$ echo $word | cut -c 1
H

$ echo $word | cut -c 2
e

$ echo $word | cut -c 1-5
Hello

“sed” for stream editor

print a file from its 2nd line to the end

This can be done via : $ sed -n '2,$ p' your_file , this is simple, however sometimes like when you are using “docker image list” command and want to get the IMAGE ID, this is what you can do :

# "docker" CLI output contains a HEADER line
$ sudo docker image list
REPOSITORY                                                TAG                 IMAGE ID            CREATED             SIZE
docker.com:4443/alma8/ubi                     latest              efa06d7afdfe        3 weeks ago         226 MB
docker.com:4443/rhel/rhel8   1.15.0              444fed57f480        6 weeks ago         367 MB

# just to print out the IMAGE ID
$ sudo docker image list | sed -n '2,$ p' | awk '{print $3}'
efa06d7afdfe
444fed57f480

# further more, if you want to delete that images
# passing the IMAGE ID to "xargs"
$ sudo docker image list | sed -n '2,$ p' | awk '{print $3}' | xargs -I {} sudo docker image rm {} 

Extract the sub string

“sed” can also help to extract the string by regex, check this sample :

$ uname -r
3.10.0-1160.102.1.el7.x86_64

# what if we only want that "el7"?
$ uname -r | sed 's/^.*\(el[0-9]\).*/\1/'
el7

x

Clean up disk space

Clean up disk space by “tmpwatch”

This CLI “tmpwatch” is from “tmpwatch” rpm package, install it if you don’t have it which can be done by : $ sudo yum install tmpwatch.

Here are some examples :

# "5d" : 5 days ago
# "--test" : dry-run, it will not really delete the files
# "--nodirs" : empty directories will NOT be removed.
$ tmpwatch 5d /tmp --test --nodirs

# except "d" for days, there're other options, e.g. "h" for hours, "m" for minutes

Clean up disk space by “find” and “xargs”

Except “tmpwatch”, there’re default common commands or system calls to do so, like “find” and “xargs”. Check the examples below.

# delete anything >= 30 days ago
$ find . -mtime +30 | xargs -I {} sudo rm -rfv {}

# only find for "directories" only
$ find . -mtime +10 -type d | xargs -I {} sudo rm -rfv {}

# find for "files"
$ find . -mtime +10 -type f | xargs -I {} sudo rm -rfv {}

# we often use "+days" because normally we want to delete old files, but just in case you want to delete recent files, then you can use this, for example <=10 days:
$ find . -mtime -10 -type f

The columns of “netstat” outputs is truncated

The output of “netstat” command, especially when it shows the full FQDN, are normally truncated like this :

$ netstat -a | more
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:sunrpc          0.0.0.0:*               LISTEN
tcp        0      0 localhost.localdo:18003 0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:ssh             0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:8089            0.0.0.0:*               LISTEN
tcp        0      0 admin00.xxx.yyyyy:41638 netauto-03.ops.s:pharos TIME_WAIT
tcp        0      0 admin00.xxx.yyyyy:58884 netauto-01.ops.s:pharos TIME_WAIT

You can either change to add “-n” so that full FQDN can be seen in IP address format, or either to use “–wide” option, e.g. $ netstat -a --wide.

Using “base64” to encode and decode

When you try to hide or you want the data can’t be seen easily or obviously, you can try “base64” command. Check this sample :

$ cat demo.sh
#!/bin/bash
echo "Hello World"

$ base64 demo.sh
IyEvYmluL2Jhc2gKZWNobyAiSGVsbG8gV29ybGQiCg==

# and then you can just send this string to others
# then use "-d" option to decode 
$ echo "IyEvYmluL2Jhc2gKZWNobyAiSGVsbG8gV29ybGQiCg==" | base64 -d
#!/bin/bash
echo "Hello World"
$