Bash Quick Start—Author Questionnaire

According to you, how has Bash enhanced programming in Linux? What is the reason for this rise in popularity?

Like other interpreted scripting languages, shell script is good for quick prototyping, or for tasks that don’t require very high performance. Its basic tools concentrate on streams of plain text, manipulating filesystem paths as strings, and simple arrays, and linking other programs together with pipes. This makes it ideal for systems-level scripts and other kinds of automation that require making programs cooperate in useful ways.

Bash in particular is popular as a shell scripting language, largely because it’s so likely already installed on a GNU/Linux system. This makes it a de facto standard; in fact, many people think Bash is the only kind of shell script.

There’s a lot to like about Bash when you know how to use it, but for a lot of users, it’s not so much popular as it is tolerated. They don’t enjoy using it, and they avoid it where they can. I hope to help change that with my book, and with my other writing on shell script, so that more users can appreciate the power of Bash.

Can you share some of the best practices of Bash shell script programming?

1. Be defensive. Handle filenames with spaces and special characters, don’t evaluate anything the user provides as code, don’t run your scripts as the root user if you can avoid it, and take particular care doing anything that’s irreversible, such as deleting files.
2. Lean on other programs. Bash can do more than a lot of people think, but its true power is in leveraging and combining what other programs can do. It’s very often a good approach to write a set of related scripts in other languages like Perl, Python, or AWK, and then to apply those scripts together in Bash.
3. Keep it simple. Bash doesn’t scale well for complex scripts. If your script is longer than 100 lines, another language might be better suited.

What makes Bash ideal for people who may have used its command line?

It’s convenient to type terse programs straight into Bash as your command interpreter to solve a systems administration task. It’s ideal for tasks like finding all the IP addresses in a set of formatted logs, or finding all the lines of a chat log where a specific word was mentioned, or looping through a list of filenames and printing the ones that correspond to executable programs. These are things that can certainly be done in almost any programming language, but they’re particularly easy in Bash.

Because your Bash shell is itself a programming language, if you write out a short program to do something useful, it’s a good idea to put it in a file to keep it for later. When you get good at doing this, it starts to feel like you’re building your own language of tools specific to a system. When you realize how powerful this approach is, you start seeing applications for it everywhere.

What makes Bash an inseparable part of Linux?

Along with the GNU C Compiler, the GNU Bourne-Again Shell was one of the first programs ported to the initial development releases of the Linux kernel back in the early 90s, so the link has been around since the very beginning. Decades later, Bash is still the default shell for new users on GNU/Linux distributions. Even with all this shared history, the kernel and the shell are still separate programs, and aren’t dependent on one another; you can run other shells on the Linux kernel, and you can run Bash on Unix kernels besides Linux.

Can you tell us some of the best Bash commands?

Here are some example command lines that do useful things that you could adapt for your own purposes, to give you a taste of what’s possible:

Show the names of all files starting with letters from a to m:

bash$ printf '%s\n' [a-m]*

Show the names of files in the current directory in uppercase:

bash$ for file in * ; do printf '%s\n' "${file^^}" ; done

Execute a command named command1, but if it doesn’t work, execute command2:

bash$ command1 || command2

These are just examples of what Bash’s built-in commands can do, which in turn is only a fraction of its real power. One of the earliest things that the book clarifies is the difference between commands or keywords that are part of the bash(1) program, like echo, for, and read, and external programs that can be called in a Bash script, like grep(1), df(1), and awk(1). The whole idea of a shell scripting language is that you can use whatever other programs you may happen to need.

What are the three most striking features that make you love Bash?

1. It’s everywhere. On any Unix-like system for which you need an interactive shell, it’s very likely that Bash will be installed; it’s probably even the default login shell. Even if it’s not there, a Bourne or POSIX compatible shell shares a lot of the same syntax, and does just fine.
2. It’s expressive. It’s true that Bash and shell script in general has a lot of dark corners, but once you know a little about how the language works, you can write short but useful programs quickly and easily.
3. It’s extensible. To me, shell script feels like defining and building a natural language for manipulating the resources relevant to any given system, perhaps more so than any other programming language family—except maybe Forth.

Can you give us a specific example (real or fictional) of how Bash can solve a problem?

There are a few simple examples above, but to get a bit more complex and specific, consider this pipeline that prints the 10 IP addresses that made the most requests of an Apache HTTPD web server, as read from its access log:

$ cut -d' ' -f1 access.log |
    sort |
    uniq -c |
    sort -k1,1nr |
    head

• cut -d' ' -f1 access.log extracts the first column from the access.log file: in this case, the IP address for the client for each request.
• The first call to sort(1) sorts all the IP addresses.
• uniq -c collects and prints a count of occurrences for each client IP.
• The second call to sort(1)—with its -k1,1nr options—sorts the lines numerically by that occurrence count, from most‐to‐fewest requests.
• Finally, head(1) prints the first 10 lines of the list.

An interesting point here is that none of these commands are part of Bash itself—they’re independent programs on the filesystem that you can use together in a shell script. It’s relatively easy to write these sorts of pipelines—but think how long it might take you to write a tool to do this in C or Java, and how many lines it would be!

What aspects of Bash do you feel are tricky to get past? What are the main challenges that anyone would face?

One of the biggest difficulties for users familiar with other programming languages is that although Bash looks very similar to a lot of languages they’ve used, it has its own syntax rules and its own way of doing things. For example, a C programmer might look at this variable assignment, giving the string value bar to foo:

foo=bar

They might then try to write their own assignment like this, with spaces around the assignment operator, which would be allowed in C:

name = "A. C. Programmer";

This doesn’t work—but to make matters worse, it gives a cryptic error message that doesn’t much help in figuring out what went wrong:

-bash: name: command not found

The language is full of unique syntactical constructs like this, so there are no shortcuts; you can’t relate much of Bash’s syntax to other programming languages, and at some point, sooner or later, you just have to learn it all from the ground up. The book helps you do just that, by examining a command line from first principles.

Can you tell us about your journey so far?

I’m a GNU/Linux systems administrator working for a internet services provider in New Zealand. I used to be a web developer. I’ve been using GNU/Linux since about 2005, and shell script was one of my favourite parts of the system from the beginning.

Your take on the IT environment with regards to innovations in IT, the knowledge and skills gap, and software development. How do you think the tech landscape is changing?

It’s interesting watching new technologies come along, and how fast everything is changing—especially in fast-moving language ecosystems like JavaScript, which I’m mostly viewing from afar. As everything continues speeding up and getting more complex, I feel like there are good career prospects in developing expertise in older technologies and tools that are still much-used but somewhat neglected by developers. If you happen to like Bash and shell script, for example, it doesn’t seem like that’ll be going away any time soon.

Can you elaborate on the key takeaways for readers?

Accept that Bash is its own thing, and don’t try to treat it the same way you would any other programming language. Learn its idioms and the typical way scripts can approach problems clearly and safely, and make an effort to figure out the particularly weird parts of its syntax. If you’re in an environment where you need to use Bash a lot, this effort will pay off many times over, and it makes the language of your command line a much more expressive environment for you to solve all kinds of problems.