Bash Tips and Tricks #1 - 10 Pastables

Abstract. This article provides some simple "pastable" Bash examples to do some useful things.

Update: This article was originally published on a previous iteration of my website. Minor changes were made to support compatibility with this site.

Let's talk about Bash. The command line shell used by default on Linux and Mac OS. This is not an introduction. This is not a tutorial. This is a collection of pastable commands that I've found useful.

Pastable #1

Renames a all files in the current directory.

for f in *; do mv $f $f.backup; done 

You can do just about anything inside that for loop, which is useful for running most any command repetitively. More generally, we can write this as:

for f in *; do <your commands here>; done

You can change * to something like *.sh to do something to all the shell scripts in the directory.

Pastable #2

Finds a file.

find . -name myfile.txt

This example searches starting from the current directory (.) and looks for a file named myfile.txt. This is great for when you know the name, or partial name, of a file but not where the file is located. You can also search starting from other locations and partial file names (which requires quotes). For example, suppose you want to find all HTML files in the /var directory.

find /var -name "*.html"

### Pastable #3

The `grep` command searches files using regular expressions.

```bash
grep "^#" myfile.sh

In this case, we're finding every line in a shell script that starts with a comment. Grep is extemely powerful and you can do a lot more with it (perhaps I'll do another post on it later). But you can always type man grep to see more options.

Pastable #4

The cut command is great for splitting output by columns.

ps -ef | cut -d ' ' -f 4

The cut command lets you split output by a particular delimiter (space in this case, but it can be any single character) and then output any fields (columns). You can output multiple fields using -f 1-3 (i.e. the first three columns). Our example outputs the process IDs for all current running processes.

Pastable #5

awk can be used similarly to cut, but it's a bit more robust.

ps -ef | awk '{ print $8 }'

Here we use awk to print the names of all the currently running processes. This post on stackexchange sheds some light on the differences between awk and cut.

Pastable #6

sed can be used as a find and replace tool.

sed 's/foo/bar/g' myfile.txt

This example will find every occurance of the word "foo" and replaces it with the word "bar" in the file myfile.txt. A great use case is if you wrote a script poorly and hardcoded a string (say a filename) all throughout the script and then you changes the name of the filename your script is referencing.

Pastable #7

The dd command is is great for dealing with binary files.

dd if=/dev/zero of=zero.bin bs=1k count=4

Create a file 4096 (4k) of null bytes.

Pastable #8

Create a file filled with 0x42.

tr '\0' '\102' < /dev/zero | dd of=42.bin bs=1k count=4

This reads in from /dev/zero, transforms zeros to 0x42 (octal 102), and uses that as the input to dd.

Pastable #9

Set a timer for yourself.

sleep 30 && echo -e "\a"

The echo -e "\a" part requires speekers, but the sleep command even by itelf is a useful timer. Let's say you're doing something and want to make sure you gave an operation enough time to complete, use something like <your command> && sleep 300 to run a command and wait 5 minutes.

Pastable #10

Sort the contents of a file and remove duplicates.

cat list.txt | sort | uniq > list.txt

cat outputs the contents of list.txt. sort, well, sorts that output. uniq is a bit more complex: it removes duplicates, but it only looks at the lines that are next to each other, so the list needs to be sorted first. Then optionally > list.txt replaces

Bonus

Type ctrl-r into your terminal to search through your previous commands. If you're accustomed to hitting the up arrow a lot to go back through previous commands, then this can speed things up a lot. It took me a while to get in the habit of using ctrl-r, but it really speeds things up if you need to go further back than a couple commands.