HPR/hpr_website
8
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Move under www to ease rsync

Browse Source
This commit is contained in:
Ken Fallon
2025-10-29 10:51:15 +01:00
parent 2bb22c7583
commit 30ad62e938
890 changed files with 0 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

458
www/eps/hpr2278/hpr2278_full_shownotes.html Executable file
View File

@@ -0,0 +1,458 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="generator" content="pandoc">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<meta name="author" content="Dave Morriss">
<title>Some supplementary Bash tips (HPR Show 2278)</title>
<style type="text/css">code{white-space: pre;}</style>
<!--[if lt IE 9]>
<script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
<link rel="stylesheet" href="http://hackerpublicradio.org/css/hpr.css">
</head>
<body id="home">
<div id="container" class="shadow">
<header>
<h1 class="title">Some supplementary Bash tips (HPR Show 2278)</h1>
<h2 class="subtitle">Pathname expansion; part 1 of 2</h2>
<h2 class="author">Dave Morriss</h2>
<hr/>
</header>
<main id="maincontent">
<article>
<header>
<h1>Table of Contents</h1>
<nav id="TOC">
<ul>
<li><a href="#expansion">Expansion</a><ul>
<li><a href="#pathname-expansion">Pathname expansion</a></li>
<li><a href="#making-test-files">Making test files</a></li>
<li><a href="#using-the-test-files">Using the test files</a></li>
<li><a href="#some-notes-about-pattern-matching">Some notes about pattern matching</a></li>
</ul></li>
<li><a href="#using-shopt-in-relation-to-pathname-expansion">Using <code>shopt</code> in relation to pathname expansion</a><ul>
<li><a href="#the-dotglob-option">The <strong><code>dotglob</code></strong> option</a></li>
<li><a href="#the-extglob-option">The <strong><code>extglob</code></strong> option</a></li>
<li><a href="#the-failglob-option">The <strong><code>failglob</code></strong> option</a></li>
<li><a href="#the-globasciiranges-option">The <strong><code>globasciiranges</code></strong> option</a></li>
<li><a href="#the-globstar-option">The <strong><code>globstar</code></strong> option</a></li>
<li><a href="#the-nocaseglob-option">The <strong><code>nocaseglob</code></strong> option</a></li>
<li><a href="#the-nullglob-option">The <strong><code>nullglob</code></strong> option</a></li>
</ul></li>
<li><a href="#conclusion">Conclusion</a></li>
<li><a href="#links">Links</a></li>
<li><a href="#manual-page-extracts">Manual Page Extracts</a><ul>
<li><a href="#expansion-1">EXPANSION</a><ul>
<li><a href="#brace-expansion">Brace Expansion</a></li>
<li><a href="#tilde-expansion">Tilde Expansion</a></li>
<li><a href="#parameter-expansion">Parameter Expansion</a></li>
<li><a href="#command-substitution">Command Substitution</a></li>
<li><a href="#arithmetic-expansion">Arithmetic Expansion</a></li>
<li><a href="#process-substitution">Process Substitution</a></li>
<li><a href="#word-splitting">Word Splitting</a></li>
<li><a href="#pathname-expansion-1">Pathname Expansion</a><ul>
<li><a href="#pattern-matching">Pattern Matching</a></li>
</ul></li>
</ul></li>
<li><a href="#shell-builtin-commands">SHELL BUILTIN COMMANDS</a></li>
</ul></li>
</ul>
</nav>
</header>
<h2 id="expansion">Expansion</h2>
<p>As we saw in the last episode <a href="http://hackerpublicradio.org/eps/hpr2045" title="Some other Bash tips">2045</a> (and others in this sub-series) there are eight types of expansion applied to the command line in the following order:</p>
<ul>
<li>Brace expansion (we looked at this subject in episode <a href="http://hackerpublicradio.org/eps/hpr1884" title="Some more Bash tips">1884</a>)</li>
<li>Tilde expansion (seen in episode <a href="http://hackerpublicradio.org/eps/hpr1903" title="Some further Bash tips">1903</a>)</li>
<li>Parameter and variable expansion (this was covered in episode <a href="http://hackerpublicradio.org/eps/hpr1648" title="Bash parameter manipulation">1648</a>)</li>
<li>Command substitution (seen in episode <a href="http://hackerpublicradio.org/eps/hpr1903" title="Some further Bash tips">1903</a>)</li>
<li>Arithmetic expansion (seen in episode <a href="http://hackerpublicradio.org/eps/hpr1951" title="Some additional Bash tips">1951</a>)</li>
<li>Process substitution (seen in episode <a href="http://hackerpublicradio.org/eps/hpr2045" title="Some other Bash tips">2045</a>)</li>
<li>Word splitting (seen in episode <a href="http://hackerpublicradio.org/eps/hpr2045" title="Some other Bash tips">2045</a>)</li>
<li>Pathname expansion (this episode and the next)</li>
</ul>
<p>This is the last topic in the (sub-) series about expansion in Bash. However, when writing the notes for this episode it became apparent that there was too much to fit into a single HPR episode. Consequently I have made it into two.</p>
<p>In this episode we will look at simple pathname expansion and some of the ways in which its behaviour can be controlled. In the next episode we’ll finish by looking at extended pattern matching. Both are included in the “<a href="#manual-page-extracts">Manual Page Extracts</a>” section at the end of the long notes.</p>
<h3 id="pathname-expansion">Pathname expansion</h3>
<p>This type of expansion is also known as <em>Filename Expansion</em> or <em>Globbing</em>. It is about the expansion of <em>wildcard</em> characters such as ‘<code>*</code>’ in filenames. You have almost certainly used it in commands like:</p>
<pre><code>ls *.txt</code></pre>
<p>The names <em>glob</em> and <em>globbing</em> have an historical origin. In the early days of Unix this type of wildcard expansion was performed by the separate program <code>/etc/glob</code>, an abbreviation of the phrase “<em>global command</em>”. Later a library function ‘<code>glob()</code>’ was provided to replace it and the name has stuck since then.</p>
<p>Operating systems other than Unix, and other environments and scripting languages also have a similar concept of <em>glob</em> patterns using wildcard characters. The actual characters often vary from those used in Bash, but the concepts are very similar. See the <a href="https://en.wikipedia.org/wiki/Glob_%28programming%29" title="Wikipedia article on &#39;glob (programming)&#39;">Wikipedia article</a> on this subject for more details.</p>
<p>Note that this process does not use <em>regular expressions</em> in the sense you will have seen in other places (such as in the HPR series called <a href="http://hackerpublicradio.org/series.php?id=90" title="Learning sed">“<em>Learning sed</em>”</a>). These <em>glob</em> patterns are older and not as sophisticated.</p>
<p>Although this process of wildcard expansion is normally used in the context of file names or paths to files, such patterns are used in other contexts as well. When we looked at parameter and variable expansion in episode <a href="http://hackerpublicradio.org/eps/hpr1648" title="Bash parameter manipulation">1648</a> we saw expressions such as:</p>
<pre><code>$ dir=&quot;/home/user/Downloads/hpr1648.ogg&quot;
$ echo ${dir##*/}
hpr1648.ogg</code></pre>
<p>Here ‘<code>*/</code>’ matches a part of the path in variable ‘<code>dir</code>’ and the operation strips it all away leaving just the terminal filename in the same way as the ‘<code>basename</code>’ command.</p>
<h3 id="making-test-files">Making test files</h3>
<p>To have some files to experiment with for this episode (and the next one) I created a series of directories and files within them:</p>
<pre><code>$ mkdir Pathname_expansion
$ cd Pathname_expansion
$ mkdir {a..z}
$ for d in {a..z}
&gt; do
&gt; touch $d/${d}{a..z}{01..50}.txt
&gt; done</code></pre>
<p>These commands do the following:</p>
<ul>
<li>Create a directory ‘<code>Pathname_expansion</code>’</li>
<li>Change directory into ‘<code>Pathname_expansion</code>’</li>
<li>Using <a href="http://hackerpublicradio.org/eps/hpr1884" title="Some more Bash tips">brace expansion</a> create the directories ‘<code>a</code>’ to ‘<code>z</code>’</li>
<li>The line beginning ‘<code>for</code>’ is the start of a multi-line command; the ‘&gt;’ means Bash is prompting for the next line:
<ul>
<li>Loop through the directories just created</li>
<li>Using ‘<code>touch</code>’ create a series of files in each one. The files begin with the letter of the directory, followed by another letter, followed by a two-digit number in the range 01-50, followed by ‘.txt’.</li>
<li>Note that to use variable ‘<code>d</code>’ as part of the filename it needs to be enclosed in ‘<code>{}</code>’ braces to separate it from the following brace expansion.</li>
</ul></li>
</ul>
<p>Each directory will therefore contain 26*50=1,300 files making a total of 33,800 (empty) files. <!-- \* --></p>
<h3 id="using-the-test-files">Using the test files</h3>
<p>So, now we need to look at how these various files could be referred to using pathname expansion.</p>
<p>According to the manual page:</p>
<blockquote>
<p>Bash scans each word for the characters ‘*’, ‘?’, and ‘[’. If one of these characters appears, then the word is regarded as a <em>pattern</em>, and replaced with an alphabetically sorted list of filenames matching the pattern.</p>
</blockquote>
<p>These pattern characters have the following meanings (see the “<a href="#manual-page-extracts">Manual Page Extracts</a>” section below under the heading “<a href="#pattern-matching">Pattern Matching</a>” for a more detailed description):</p>
<dl>
<dt><b>*</b></dt>
<dd><p>Matches any string, including the null string.</p>
</dd>
<dt><strong>?</strong></dt>
<dd><p>Matches any single character.</p>
</dd>
<dt><b>[…]</b></dt>
<dd><p>Matches any one of the enclosed characters, such as <code>[abc]</code>.</p>
<p>A pair of characters separated by a hyphen (such as <code>[a-z]</code>) denotes a <em>range expression</em>; any character that falls between those two characters, inclusive is matched.<a href="#fn1" class="footnoteRef" id="fnref1"><sup>1</sup></a></p>
<p>If the first character following the ‘<code>[</code>’ is a ‘<code>!</code>’ (exclamation mark) or a ‘<code>^</code>’ (circumflex) then any character not enclosed is matched. Note that ‘<code>!</code>’ is the POSIX standard, making ‘<code>^</code>’ non-standard.</p>
<p>A ‘<code>-</code>’ (hyphen) may be matched by including it as the first or last character in the set, such as <code>[-a-z]</code> or <code>[a-z-]</code> meaning any letter from the range expression as well as the hyphen.</p>
<p>A ‘<code>]</code>’ (close square bracket) may be matched by including it as the first character in the set, such as <code>[]a-z]</code>.</p>
<p>Other <em>character classes</em> may be used within these square brackets, where a class has the form <code>[:class:]</code>, such as <code>[[:alnum:]]</code> which has the same meaning as <code>[a-zA-Z0-9]</code>. See the more detailed manual page extracts at the end of this document.</p>
</dd>
</dl>
<p>To refer to all files in the directory ‘<code>a</code>’ which have ‘<code>a</code>’ as the second letter we could use the following pattern:</p>
<pre><code>$ ls a/?a*</code></pre>
<p>Here the question mark ‘<code>?</code>’ means any character (though we know that all the files begin with ‘<code>a</code>’ in this directory). This is followed by a letter ‘<code>a</code>’ meaning the second letter must be ‘<code>a</code>’. Finally there is an asterisk ‘<code>*</code>’ which means that the rest of the filename can be anything.</p>
<p>This command returns 50 filenames like this (only the first two lines are shown):</p>
<pre><code>$ ls -w 60 a/?a*
a/aa01.txt a/aa11.txt a/aa21.txt a/aa31.txt a/aa41.txt
a/aa02.txt a/aa12.txt a/aa22.txt a/aa32.txt a/aa42.txt
...</code></pre>
<p>Note that the use of ‘<code>-w 60</code>’ restricts the number of columns produced by <code>ls</code>.</p>
<h3 id="some-notes-about-pattern-matching">Some notes about pattern matching</h3>
<p>As already mentioned, there is a certain resemblance between these patterns and <em>regular expressions</em>, which you may have encountered in other HPR episodes such as <a href="http://hackerpublicradio.org/series.php?id=90" title="Learning sed"><em>Learning sed</em></a> and <a href="http://hackerpublicradio.org/series.php?id=94" title="Learning Awk"><em>Learning Awk</em></a>. The two should not be confused, regular expressions are far more powerful, but are not available in Bash in the same context.</p>
<p>The expansion of these patterns takes place on the command line, resulting in an alphabetical list of pathnames, and these are presented to the command. For example, the <code>echo</code> command may be used:</p>
<pre><code>$ echo a/?a0*
a/aa01.txt a/aa02.txt a/aa03.txt a/aa04.txt a/aa05.txt a/aa06.txt a/aa07.txt a/aa08.txt a/aa09.txt</code></pre>
<p>Here the pattern ‘<code>a/?a0*</code>’ was used, meaning files in directory ‘a’ starting with any character, followed by an ‘a’, a zero and then any number of further characters. This was expanded by the Bash shell and the nine pathnames were passed to <code>echo</code> which printed them.</p>
<p>It might help to demonstrate this more clearly by using arrays (covered to some extent in the episode entitled <a href="http://hackerpublicradio.org/eps/hpr1648" title="Bash parameter manipulation"><em>Bash parameter manipulation</em></a>):</p>
<pre><code>$ vec=(a/?a0*)
$ echo ${#vec[@]}
9
$ echo ${vec[@]}
a/aa01.txt a/aa02.txt a/aa03.txt a/aa04.txt a/aa05.txt a/aa06.txt a/aa07.txt a/aa08.txt a/aa09.txt</code></pre>
<p>Here the array called ‘<code>vec</code>’ is filled with the result of the pathname expansion using the same pattern as before. When we use the substitution syntax ‘<code>${#vec[@]}</code>’ we get the number of elements in the array, and ‘<code>${vec[@]}</code>’ returns all of the elements which are printed by <code>echo</code>.</p>
<p>These pattern expansions do not occur when enclosed in single or double quotation marks. Such a pattern is treated simply as a verbatim string:</p>
<pre><code>$ echo &quot;a/?a0*&quot;
a/?a0*</code></pre>
<p>Note also that if the pattern does not end with a wildcard then it implies that the final part exactly matches the end of the file name:</p>
<pre><code>$ echo a/aa1*tx
a/a1*tx
$ echo a/aa1*txt
a/aa10.txt a/aa11.txt a/aa12.txt a/aa13.txt a/aa14.txt a/aa15.txt a/aa16.txt a/aa17.txt a/aa18.txt a/aa19.txt</code></pre>
<p>All files end with ‘txt’, so ending the pattern with ‘tx’ matches nothing.</p>
<p>Later in this episode we will look in more detail at how the expansion process just returns the pattern if there are no matches.</p>
<!-- More examples? -->
<h2 id="using-shopt-in-relation-to-pathname-expansion">Using <code>shopt</code> in relation to pathname expansion</h2>
<p>There are a number of Bash options that affect the way that pathname expansion works. These are referred to in detail in the manual page extracts at the end of these notes.</p>
<p>The <code>shopt</code> command is built into the Bash shell. Typing it on its own results in a list of all of the options and their settings:</p>
<pre><code>$ shopt
autocd off
cdable_vars off
cdspell off
...</code></pre>
<p>(The rest have been omitted.)</p>
<p>Typing <code>shopt</code> with the name of an option returns its current setting:</p>
<pre><code>$ shopt dotglob
dotglob off</code></pre>
<p>To turn on an option use <code>shopt -s</code> followed by the option name (‘s’ stands for ‘set’):</p>
<pre><code>$ shopt -s dotglob</code></pre>
<p>Turning it off is achieved with <code>shopt -u</code> (‘u’ stands for ‘unset’):</p>
<pre><code>$ shopt -u dotglob</code></pre>
<p>The status of the settings can be reported in a form that can be saved and used as commands by specifying the <code>-p</code> option:</p>
<pre><code>$ shopt -p dotglob failglob
shopt -u dotglob
shopt -u failglob</code></pre>
<p>We will look at a subset of the settings controlled by <code>shopt</code>. This subset consists of the settings which are of relevance to pathname expansion.</p>
<h3 id="the-dotglob-option">The <strong><code>dotglob</code></strong> option</h3>
<p>This option controls whether files beginning with a dot (‘.’) are returned by pathname expansion. Normally, they are not.</p>
<p>To demonstrate this we first create a file with a name beginning with a dot:</p>
<pre><code>$ touch a/.dotfile</code></pre>
<p>Such files are called <em>hidden</em> because many parts of the operating system do not show them unless requested.</p>
<p>Normally trying to find such a file using a pathname with wildcards fails:</p>
<pre><code>$ ls a/*dot*
ls: cannot access &#39;a/*dot*&#39;: No such file or directory
$ ls a/?dot*
ls: cannot access &#39;a/?dot*&#39;: No such file or directory
$ ls a/[.]dot*
ls: cannot access &#39;a/[.]dot*&#39;: No such file or directory</code></pre>
<p>You might think that adding the <code>-a</code> option to <code>ls</code> (which shows <em>hidden</em> files) might solve the problem. It does not. The issue is that the target file is not returned by the expansion, so the <code>ls</code> command is simply given the pattern, which it treats as a filename and there is no file called “<em>asterisk-d-o-t-asterisk</em>” or any of the others with literal wildcards.</p>
<p>However, if <code>dotglob</code> is on then the file becomes visible:</p>
<pre><code>$ shopt -s dotglob
$ ls a/*dot*
a/.dotfile</code></pre>
<p>Of course, the file is visible to <code>ls</code> if the <code>-a</code> option is used (and <code>dotglob</code> is off), and no pathname expansion is used. However, in this case all 1300 other files in the directory would be listed.</p>
<p>We’ll list the filenames in one column (-1) and view just the last 3 to demonstrate this:</p>
<pre><code>$ ls -1 -a a | tail -3
az49.txt
az50.txt
.dotfile</code></pre>
<p>As as Unix newbie I struggled with this “<em>dotfile</em>” issue a lot. I hope this has helped to clarify things for you.</p>
<h3 id="the-extglob-option">The <strong><code>extglob</code></strong> option</h3>
<p>This option controls whether extended pattern matching features are enabled or not. We will look at these in the next episode.</p>
<h3 id="the-failglob-option">The <strong><code>failglob</code></strong> option</h3>
<p>This option controls whether an error is produced when a pattern fails to match filenames during pathname expansion.</p>
<p>The example shows that when <code>failglob</code> is on the failure of the match is detected early and the command aborted, otherwise the failed pattern is passed to the <code>ls</code> command.</p>
<pre><code>$ shopt -s failglob
$ ls a/aa50*
a/aa50.txt
$ ls a/aa51*
-bash: no match: a/aa51*
$ shopt -u failglob
$ ls a/aa51*
ls: cannot access &#39;a/aa51*&#39;: No such file or directory</code></pre>
<p>Note that turning on the <code>failglob</code> option has other effects that might not be very desirable, such as on Tab completion. Use with caution.</p>
<h3 id="the-globasciiranges-option">The <strong><code>globasciiranges</code></strong> option</h3>
<p>Setting this option on disables the use of the collating sequence of the current locale, and reverts to traditional ASCII. It is relevant to bracket expressions like <code>[a-z]</code>.</p>
<pre><code>$ mkdir test
$ cd test
$ touch á
$ touch b
$ shopt -s globasciiranges
$ ls [a-b]
b
$ shopt -u globasciiranges
$ ls [a-b]
á b
$ cd -</code></pre>
<p>Setting <code>globasciiranges</code> makes the file called ‘<code>á</code>’ disappear.</p>
<h3 id="the-globstar-option">The <strong><code>globstar</code></strong> option</h3>
<p>When this option is on the pattern ‘<code>**</code>’ causes recursive scanning of directories when pattern matching.</p>
<p>To demonstrate this we will create some extra directories and files:</p>
<pre><code>$ mkdir -p dir1/dir2/dir3
$ touch dir1/dir2/dir3/{test.tmp,README} dir1/dir2/{test2.tmp,test2.dat} dir1/test3.tmp
$ tree dir1/
dir1/
├── dir2
│   ├── dir3
│   │   ├── README
│   │   └── test.tmp
│   ├── test2.dat
│   └── test2.tmp
└── test3.tmp
2 directories, 5 files</code></pre>
<p>Note the use of <code>mkdir -p</code> to create all directories at once, the multiple arguments to <code>touch</code> that use <em>brace expansion</em> and the <code>tree</code> command that draws a diagram of the directory structure.</p>
<p>Now we can list files in this tree structure by using the ‘<code>**</code>’ pattern. We will use <code>echo</code> again since <code>ls</code> will show all files in directories if their names are returned after expansion:</p>
<pre><code>$ shopt -s globstar
$ echo **/*.tmp
dir1/dir2/dir3/test.tmp dir1/dir2/test2.tmp dir1/test3.tmp</code></pre>
<p>Note that if ‘<code>**</code>’ is followed by a ‘/’ only directories are matched.</p>
<pre><code>$ echo dir1/**/
dir1/ dir1/dir2/ dir1/dir2/dir3/</code></pre>
<p>Here, the <code>ls</code> command receives the directory names then lists their contents:</p>
<pre><code>$ ls dir1/**/
dir1/:
dir2 test3.tmp
dir1/dir2/:
dir3 test2.dat test2.tmp
dir1/dir2/dir3/:
README test.tmp</code></pre>
<p>With <code>globstar</code> turned off we cannot recurse through the directory structure looking for files and ‘<code>**</code>’ has no special meaning:</p>
<pre><code>$ shopt -u globstar
$ echo **/*.tmp
dir1/test3.tmp</code></pre>
<p>Learning to use the <code>find</code> command can be a better solution to the problems of finding files in a directory hierarchy:</p>
<pre><code>$ find dir1 -name &quot;*.tmp&quot;
dir1/dir2/dir3/test.tmp
dir1/dir2/test2.tmp
dir1/test3.tmp</code></pre>
<h3 id="the-nocaseglob-option">The <strong><code>nocaseglob</code></strong> option</h3>
<p>Normally pathname expansion is case sensitive. Setting the <code>nocaseglob</code> option turns off case-sensitivity.</p>
<p>With <code>nocaseglob</code> off the pattern matches nothing:</p>
<pre><code>$ echo a/AA0*
a/AA0*</code></pre>
<p>With the option on the same pattern matches files:</p>
<pre><code>$ shopt -s nocaseglob
$ echo a/AA0*
a/aa01.txt a/aa02.txt a/aa03.txt a/aa04.txt a/aa05.txt a/aa06.txt a/aa07.txt a/aa08.txt a/aa09.txt</code></pre>
<p>However, the directory does not match in a case-insensitive way because it is not part of a pattern:</p>
<pre><code>$ echo A/AA0*
A/AA0*</code></pre>
<p>When a pattern is used for the directory then a case-insensitive match works:</p>
<pre><code>$ echo [A]/AA0*
a/aa01.txt a/aa02.txt a/aa03.txt a/aa04.txt a/aa05.txt a/aa06.txt a/aa07.txt a/aa08.txt a/aa09.txt
$ echo ?/AA0*
a/aa01.txt a/aa02.txt a/aa03.txt a/aa04.txt a/aa05.txt a/aa06.txt a/aa07.txt a/aa08.txt a/aa09.txt</code></pre>
<h3 id="the-nullglob-option">The <strong><code>nullglob</code></strong> option</h3>
<p>As we saw when discussing <code>dotglob</code>, a pattern that matches nothing is returned intact, which might result in a command treating it as a pathname.</p>
<p>The <code>nullglob</code> option, when on, results in a null string being returned in such cases.</p>
<pre><code>$ ls a/*dot*
ls: cannot access &#39;a/*dot*&#39;: No such file or directory
$ shopt -s nullglob
$ echo &quot;[&quot; a/*dot* &quot;]&quot;
[ ]
$ shopt -u nullglob
$ echo &quot;[&quot; a/*dot* &quot;]&quot;
[ a/*dot* ]</code></pre>
<p>Here the <code>ls</code> command used before is demonstrated showing the pattern being returned when the match fails. Then <code>nullglob</code> is turned on and <code>echo</code> is used to demonstrate the null string being returned. We use (quoted) brackets to show this. When <code>nullglob</code> is off then the pattern is returned as before.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Pathname expansion and a knowledge of the patterns Bash uses is very important for effective use of the Bash command line or for writing Bash scripts. The various options controlled by <code>shopt</code> are less critical, with the exception of <code>dotglob</code> perhaps.</p>
<p>In the next (and final) episode about expansion we will look at other factors controlling expansion and will examine the <em>extended pattern matching</em> operators.</p>
<h2 id="links">Links</h2>
<!-- \_ -->
<ul>
<li>Previous shows in this series
<ol>
<li><a href="http://hackerpublicradio.org/eps/hpr1648">HPR episode 1648 “<em>Bash parameter manipulation</em>”</a> (2014-11-26)</li>
<li><a href="http://hackerpublicradio.org/eps/hpr1843">HPR episode 1843 “<em>Some Bash tips</em>”</a> (2015-08-26)</li>
<li><a href="http://hackerpublicradio.org/eps/hpr1884">HPR episode 1884 “<em>Some more Bash tips</em>”</a> (2015-10-22)</li>
<li><a href="http://hackerpublicradio.org/eps/hpr1903">HPR episode 1903 “<em>Some further Bash tips</em>”</a> (2015-11-18)</li>
<li><a href="http://hackerpublicradio.org/eps/hpr1951">HPR episode 1951 “<em>Some additional Bash tips</em>”</a> (2016-01-25)</li>
<li><a href="http://hackerpublicradio.org/eps/hpr2045">HPR episode 2045 “<em>Some other Bash tips</em>”</a> (2016-06-03)</li>
</ol></li>
</ul>
<!-- - -->
<ul>
<li>Other HPR series referenced:
<ul>
<li><a href="http://hackerpublicradio.org/series.php?id=90">“<em>Learning sed</em>”</a> series on HPR</li>
<li><a href="http://hackerpublicradio.org/series.php?id=94">“<em>Learning Awk</em>”</a> series on HPR <!-- - --></li>
</ul></li>
<li>Wikipedia article on <a href="https://en.wikipedia.org/wiki/Glob_%28programming%29"><em>glob patterns</em></a></li>
<li>Advanced Bash Scripting Guide: <a href="http://www.tldp.org/LDP/abs/html/globbingref.html">“<em>Globbing</em>”</a></li>
<li>Article on <a href="http://mywiki.wooledge.org/glob"><em>Greg’s Wiki</em></a> entitled “<em>Globs</em>”</li>
</ul>
<hr />
<!-- {{{ -->
<h1 id="manual-page-extracts">Manual Page Extracts</h1>
<h2 id="expansion-1">EXPANSION</h2>
<p>Expansion is performed on the command line after it has been split into words. There are seven kinds of expansion performed: <em>brace expansion</em>, <em>tilde expansion</em>, <em>parameter and variable expansion</em>, <em>command substitution</em>, <em>arithmetic expansion</em>, <em>word splitting</em>, and <em>pathname expansion</em>.</p>
<p>The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and pathname expansion.</p>
<p>On systems that can support it, there is an additional expansion available: <em>process substitution</em>. This is performed at the same time as tilde, parameter, variable, and arithmetic expansion and command substitution.</p>
<p>Only brace expansion, word splitting, and pathname expansion can change the number of words of the expansion; other expansions expand a single word to a single word. The only exceptions to this are the expansions of “<strong>$@</strong>” and “<strong>${name[@]}</strong>” as explained above (see <strong>PARAMETERS</strong>).</p>
<h3 id="brace-expansion">Brace Expansion</h3>
<p>See the notes for HPR show <a href="http://hackerpublicradio.org/eps/hpr1884" title="Some more Bash tips">1884</a>.</p>
<h3 id="tilde-expansion">Tilde Expansion</h3>
<p>See the notes for HPR show <a href="http://hackerpublicradio.org/eps/hpr1903" title="Some further Bash tips">1903</a>.</p>
<h3 id="parameter-expansion">Parameter Expansion</h3>
<p>See the notes for HPR show <a href="http://hackerpublicradio.org/eps/hpr1648" title="Bash parameter manipulation">1648</a>.</p>
<h3 id="command-substitution">Command Substitution</h3>
<p>See the notes for HPR show <a href="http://hackerpublicradio.org/eps/hpr1903" title="Some further Bash tips">1903</a>.</p>
<h3 id="arithmetic-expansion">Arithmetic Expansion</h3>
<p>See the notes for HPR show <a href="http://hackerpublicradio.org/eps/hpr1951" title="Some additional Bash tips">1951</a>.</p>
<h3 id="process-substitution">Process Substitution</h3>
<p>See the notes for HPR show <a href="http://hackerpublicradio.org/eps/hpr2045" title="Some other Bash tips">2045</a>.</p>
<h3 id="word-splitting">Word Splitting</h3>
<p>See the notes for HPR show <a href="http://hackerpublicradio.org/eps/hpr2045" title="Some other Bash tips">2045</a>.</p>
<h3 id="pathname-expansion-1">Pathname Expansion</h3>
<p>After word splitting, unless the <strong>-f</strong> option has been set, <strong>bash</strong> scans each word for the characters <b>*</b>, <b>?</b>, and <b>[</b>. If one of these characters appears, then the word is regarded as a <em>pattern</em>, and replaced with an alphabetically sorted list of filenames matching the pattern (see <strong>Pattern Matching</strong> below). If no matching filenames are found, and the shell option <strong>nullglob</strong> is not enabled, the word is left unchanged. If the <strong>nullglob</strong> option is set, and no matches are found, the word is removed. If the <strong>failglob</strong> shell option is set, and no matches are found, an error message is printed and the command is not executed. If the shell option <strong>nocaseglob</strong> is enabled, the match is performed without regard to the case of alphabetic characters. Note that when using range expressions like [a-z] (see below), letters of the other case may be included, depending on the setting of <strong>LC_COLLATE</strong>. When a pattern is used for pathname expansion, the character “.” at the start of a name or immediately following a slash must be matched explicitly, unless the shell option <strong>dotglob</strong> is set. When matching a pathname, the slash character must always be matched explicitly. In other cases, the “.” character is not treated specially. See the description of <strong>shopt</strong> below under <strong>SHELL BUILTIN COMMANDS</strong> for a description of the <strong>nocaseglob</strong>, <strong>nullglob</strong>, <strong>failglob</strong>, and <strong>dotglob</strong> shell options.</p>
<p>The <strong>GLOBIGNORE</strong> shell variable may be used to restrict the set of filenames matching a pattern. If <strong>GLOBIGNORE</strong> is set, each matching filename that also matches one of the patterns in <strong>GLOBIGNORE</strong> is removed from the list of matches. The filenames “.” and “..” are always ignored when <strong>GLOBIGNORE</strong> is set and not null. However, setting <strong>GLOBIGNORE</strong> to a non-null value has the effect of enabling the <strong>dotglob</strong> shell option, so all other file‐ names beginning with a “.” will match. To get the old behavior of ignoring filenames beginning with a “.”, make “.*&quot; one of the patterns in <strong>GLOBIGNORE</strong>. The <strong>dotglob</strong> option is disabled when <strong>GLOBIGNORE</strong> is unset.</p>
<h4 id="pattern-matching">Pattern Matching</h4>
<p>Any character that appears in a pattern, other than the special pattern characters described below, matches itself. The NUL character may not occur in a pattern. A backslash escapes the following character; the escaping backslash is discarded when matching. The special pattern characters must be quoted if they are to be matched literally.</p>
<p>The special pattern characters have the following meanings:</p>
<dl>
<dt><b>*</b></dt>
<dd><p>Matches any string, including the null string. When the <strong>globstar</strong> shell option is enabled, and <b>*</b> is used in a pathname expansion context, two adjacent <b>*</b>s used as a single pattern will match all files and zero or more directories and subdirectories. If followed by a <b>/</b>, two adjacent <b>*</b>s will match only directories and subdirectories.</p>
</dd>
<dt><strong>?</strong></dt>
<dd><p>Matches any single character.</p>
</dd>
<dt><b>[…]</b></dt>
<dd><p>Matches any one of the enclosed characters. A pair of characters separated by a hyphen denotes a <em>range expression</em>; any character that falls between those two characters, inclusive, using the current locale’s collating sequence and character set, is matched. If the first character following the <strong>[</strong> is a <strong>!</strong> or a <strong>^</strong> then any character not enclosed is matched. The sorting order of characters in range expressions is determined by the current locale and the values of the <strong>LC_COLLATE</strong> or <strong>LC_ALL</strong> shell variables, if set. To obtain the traditional interpretation of range expressions, where <strong>[a-d]</strong> is equivalent to <strong>[abcd]</strong>, set value of the <strong>LC_ALL</strong> shell variable to <strong>C</strong>, or enable the <strong>globasciiranges</strong> shell option. A <strong>-</strong> may be matched by including it as the first or last character in the set. A <strong>]</strong> may be matched by including it as the first character in the set.</p>
<p>Within <b>[</b> and <b>]</b>, character classes can be specified using the syntax <b>[:</b><em>class</em><b>:]</b>, where <em>class</em> is one of the following classes defined in the POSIX standard: <b>alnum alpha ascii blank cntrl digit graph lower print punct space upper word xdigit</b> A character class matches any character belonging to that class. The <b>word</b> character class matches letters, digits, and the character _.</p>
<p>Within <b>[</b> and <b>]</b>, an <em>equivalence class</em> can be specified using the syntax <b>[=</b><em>c</em><b>=]</b>, which matches all characters with the same collation weight (as defined by the current locale) as the character <em>c.</em></p>
<p>Within <b>[</b> and <b>]</b>, the syntax <b>[.</b><em>symbol</em><b>.]</b> matches the collating symbol <em>symbol</em>.</p>
</dd>
</dl>
<p>If the <b>extglob</b> shell option is enabled using the <b>shopt</b> builtin, several extended pattern matching operators are recognized. In the following description, a <em>pattern-list</em> is a list of one or more patterns separated by a <b>|</b>. Composite patterns may be formed using one or more of the following sub-patterns:</p>
<dl>
<dt><b>?(</b><em>pattern-list</em><b>)</b></dt>
<dd><p>Matches zero or one occurrence of the given patterns</p>
</dd>
<dt><b><em>(</b></em>pattern-list*<b>)</b></dt>
<dd><p>Matches zero or more occurrences of the given patterns</p>
</dd>
<dt><b>+(</b><em>pattern-list</em><b>)</b></dt>
<dd><p>Matches one or more occurrences of the given patterns</p>
</dd>
<dt><b>@(</b><em>pattern-list</em><b>)</b></dt>
<dd><p>Matches one of the given patterns</p>
</dd>
<dt><b>!(</b><em>pattern-list</em><b>)</b></dt>
<dd><p>Matches anything except one of the given patterns</p>
</dd>
</dl>
<hr />
<h2 id="shell-builtin-commands">SHELL BUILTIN COMMANDS</h2>
<p>This is an extract relating to the <code>shopt</code> builtin. Only the options relating to pathname expansion are included. For the full list refer to the Bash manual page.</p>
<dl>
<dt><b>shopt [-pqsu] [-o] [optname …]</b></dt>
<dd><p>Toggle the values of settings controlling optional shell behavior. The settings can be either those listed below, or, if the <b>-o</b> option is used, those available with the <b>-o</b> option to the set builtin command. With no options, or with the <b>-p</b> option, a list of all settable options is displayed, with an indication of whether or not each is set. The <b>-p</b> option causes output to be displayed in a form that may be reused as input. Other options have the following meanings:</p>
<pre><code>-s Enable (set) each optname.
-u Disable (unset) each optname.
-q Suppresses normal output (quiet mode); the return status
indicates whether the optname is set or unset. If multiple optname
arguments are given with -q, the return status is zero if all optnames
are enabled; non-zero otherwise.
-o Restricts the values of optname to be those defined for the -o
option to the set builtin.</code></pre>
<p>If either <b>-s</b> or <b>-u</b> is used with no optname arguments, shopt shows only those options which are set or unset, respectively. Unless otherwise noted, the shopt options are disabled (unset) by default.</p>
<p>The return status when listing options is zero if all optnames are enabled, non-zero otherwise. When setting or unsetting options, the return status is zero unless an optname is not a valid shell option.</p>
<p>The list of shopt options is:</p>
<pre><code>dotglob If set, bash includes filenames beginning with a `.&#39; in the
results of pathname expansion.
extglob If set, the extended pattern matching features described above
under Pathname Expansion are enabled.
failglob
If set, patterns which fail to match filenames during pathname
expansion result in an expansion error.
globasciiranges
If set, range expressions used in pattern matching bracket
expressions (see Pattern Matching above) behave as if in the
traditional C locale when performing comparisons. That is, the
current locale&#39;s collating sequence is not taken into account,
so b will not collate between A and B, and upper-case and
lower-case ASCII characters will collate together.
globstar
If set, the pattern ** used in a pathname expansion context
will match all files and zero or more directories and
subdirectories. If the pattern is followed by a /, only
directories and subdirectories match.
nocaseglob
If set, bash matches filenames in a case-insensitive fashion
when performing pathname expansion (see Pathname Expansion
above).
nullglob
If set, bash allows patterns which match no files (see
Pathname Expansion above) to expand to a null string, rather
than themselves.</code></pre>
</dd>
</dl>
<!-- }}} -->
<section class="footnotes">
<hr />
<ol>
<li id="fn1"><p>The simple concept of a range expression is complicated considerably by the fact that since it was invented many more character sets than plain ASCII have been added. The way in which such ranges are interpreted depends on the current <em>LOCALE</em>. See the <a href="#manual-page-extracts">Manual Page Extracts</a> section for details.<a href="#fnref1">↩</a></p></li>
</ol>
</section>
</article>
</main>
</div>
</body>
</html>