You’ve learned how to create scripts, use arguments, and build for loops. Now, let’s take a look at some more basic commands, text file manipulation, and redirecting input and output to files and other commands.
Some Basic Useful Commands
We’ve already given you a basic breakdown of shell scripts and an example-based outline of for loops, but be sure to check out those articles if you’ve missed our shell scripting guide thus far.
The command-line is wonderful for many reasons, and redirection is one of the most prominent. If you had to make note of and reproduce the output of every command in order to take further action or use that for something else, then we’d all have gone crazy long ago. Redirection lets us use that output and save it or immediately use it as an input for another command. We can also use files as inputs for other commands.
Before we go on, let’s cover some basic commands which can be of use in many different places.
echo – This command simply prints (displays) the entirety of its argument on the command-line as output
echo argument with spaces
As you can see, special characters need to be “escaped” so they are treated normally. This is done by using a backslash (\) in front of the character. It’s a better idea to use quotes. The echo command also works with variables.
As you can see, single and double quotes behave differently. For more information, check out What’s the Difference Between Single and Double Quotes in the Bash Shell?
cat – This command displays the contents of text files as output.
cat file_to_be_read
Let’s say we create this text file in nano:
When we use the cat command on the file, we can see it’s output.
grep – This is one of the most powerful and useful commands available to you in Linux. It stands for Global/Regular Expression Print. It looks through a file and prints any line that matches a particular pattern. Because this pattern is based on “regular expression,” a concise line can yield a multitude of patterns to be matched. For not, though, you can enter a tern for searching.
grep pattern file
I assure you, grep can do more, but for now let’s stick to the easier stuff.
Redirecting Outputs
To redirect the output of a command to a file, we make use of a special character, the greater-than symbol (>).
Let’s change our list up, shall we? Enter the following command:
echo pepperoni > list
You can see that echo doesn’t display the line anymore, and when we look at the contents of the “list” file, we see what we echoed in there.
Also take note that the previous contents of “list” were removed. Try it again:
This can be useful when you want to reuse a file, but often we just want to add to an existing file. For this, we use two consecutive greater-than symbols:
echo yellow peppers >> list
Easy! Let’s use this command to create a larger list, shall we?
There we go. I think you can see why so many geeks use the command-line to make to-do lists and the like, but it gets even better.
Let’s take the output of a command and put it into a file:
ls –al / > ~/rootlist
Making lists of files, editing them down, and then running commands on the ones you want has never been simpler. And, while we’re doing these basic functions in the command-line, these work well in scripts, too.
Piping, or Chaining
Piping is so named because it uses the pipe, (| ; shared with the \ key on most keyboards). Essentially, it takes the output of one command and directly feeds it to another. You can create long chains of commands to get a very specific desired output this way, and it’s very convenient for commands like grep.
It acts a lot like “>” except it can be chained multiple times and its effect is more general in that it doesn’t need to go through a text file.
As you can see, grep is case-sensitive. You can use the “-i” flag to make it ignore case.
Redirecting Inputs
You can also take inputs from files for commands by using the less-than symbol (<).
cat < list
“That’s not any different from using an argument!” you might say. Well, you’d be correct in this case. Where redirection of input really comes in handy is in chaining commands together.
Let’s say we want to filter any word that has “pep” in it from our current “list” file into a new file called “revisions”.
grep pep < list > revisions
Let’s redo this command, and add some sorting.
grep pep < list | sort > revisions
This will use “pep” as the search term from the input file “list”, sort it in alphabetical order (all upper-case terms followed by all lower-case terms), then output it into the “revisions” file.
To illustrate the sort command, let’s look at the following example:
As you can see, adding the “-f” flag to the sort command allows you to ignore case. This makes it easy for us to alphabetize lines in text files and ignore capitalization when it doesn’t matter.
A Simple Script
Let’s create a script that has the following form:
script searchterm listfile
It will take the term and use grep to search through a list file, sort the results, and then output them to another file.
Here’s the directory that we’ll be testing the script in:
And, we can create a list of what in here, then run the script.
There you go! The more you learn the rules of regular expressions, the more accurately you can put together a search command. And, anything that is valid in quotes can be substituted for your first argument!
As far as sorting goes, you can do more than just sort alphabetically. Take a look at the man page for some of the following commands:
- tsort – a more advanced topological sorting function
- tr – lets you map specific characters to other characters, and transcribe between them.
- uniq – removes any non-unique (read: duplicate)
- awk – a really advanced text processing language/function that can be used to separate fields in filenames
- cut, paste/join – commands useful for isolating fields from text files and adding new data into columns
- look – searches like grep does, but uses a dictionary file (that can be user-specified) for the lookup
- wc – lets you get word count, line count, character count, and more
We took a look at some more basics today that can be as useful on the command-line as in scripts. Text-based data is often at the heart of things we use daily, so being able to work with it, search it, and manipulate it is key.
What are some of your favorite scripts? Have any special-use scripts for text-based files? Share what you know in the comments!