495 lines
34 KiB
HTML
Executable File
495 lines
34 KiB
HTML
Executable File
<!DOCTYPE html>
|
||
<html>
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="generator" content="pandoc">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||
<meta name="author" content="Dave Morriss">
|
||
<title>Useful Bash functions - part 4 (HPR Show 2483)</title>
|
||
<style type="text/css">code{white-space: pre;}</style>
|
||
<!--[if lt IE 9]>
|
||
<script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
|
||
<![endif]-->
|
||
<style type="text/css">
|
||
div.sourceCode { overflow-x: auto; }
|
||
table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode {
|
||
margin: 0; padding: 0; vertical-align: baseline; border: none; }
|
||
table.sourceCode { width: 100%; line-height: 100%; }
|
||
td.lineNumbers { text-align: right; padding-right: 4px; padding-left: 4px; color: #aaaaaa; border-right: 1px solid #aaaaaa; }
|
||
td.sourceCode { padding-left: 5px; }
|
||
code > span.kw { color: #007020; font-weight: bold; } /* Keyword */
|
||
code > span.dt { color: #902000; } /* DataType */
|
||
code > span.dv { color: #40a070; } /* DecVal */
|
||
code > span.bn { color: #40a070; } /* BaseN */
|
||
code > span.fl { color: #40a070; } /* Float */
|
||
code > span.ch { color: #4070a0; } /* Char */
|
||
code > span.st { color: #4070a0; } /* String */
|
||
code > span.co { color: #60a0b0; font-style: italic; } /* Comment */
|
||
code > span.ot { color: #007020; } /* Other */
|
||
code > span.al { color: #ff0000; font-weight: bold; } /* Alert */
|
||
code > span.fu { color: #06287e; } /* Function */
|
||
code > span.er { color: #ff0000; font-weight: bold; } /* Error */
|
||
code > span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
|
||
code > span.cn { color: #880000; } /* Constant */
|
||
code > span.sc { color: #4070a0; } /* SpecialChar */
|
||
code > span.vs { color: #4070a0; } /* VerbatimString */
|
||
code > span.ss { color: #bb6688; } /* SpecialString */
|
||
code > span.im { } /* Import */
|
||
code > span.va { color: #19177c; } /* Variable */
|
||
code > span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
|
||
code > span.op { color: #666666; } /* Operator */
|
||
code > span.bu { } /* BuiltIn */
|
||
code > span.ex { } /* Extension */
|
||
code > span.pp { color: #bc7a00; } /* Preprocessor */
|
||
code > span.at { color: #7d9029; } /* Attribute */
|
||
code > span.do { color: #ba2121; font-style: italic; } /* Documentation */
|
||
code > span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
|
||
code > span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
|
||
code > span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
|
||
</style>
|
||
<link rel="stylesheet" href="http://hackerpublicradio.org/css/hpr.css">
|
||
</head>
|
||
|
||
<body id="home">
|
||
<div id="container" class="shadow">
|
||
<header>
|
||
<h1 class="title">Useful Bash functions - part 4 (HPR Show 2483)</h1>
|
||
<h2 class="author">Dave Morriss</h2>
|
||
<hr/>
|
||
</header>
|
||
|
||
<main id="maincontent">
|
||
<article>
|
||
<header>
|
||
<h1>Table of Contents</h1>
|
||
<nav id="TOC">
|
||
<ul>
|
||
<li><a href="#overview">Overview</a></li>
|
||
<li><a href="#the-range_parse-function">The <code>range_parse</code> function</a><ul>
|
||
<li><a href="#algorithm">Algorithm</a></li>
|
||
<li><a href="#analysis-of-function">Analysis of function</a></li>
|
||
<li><a href="#explanations">Explanations</a></li>
|
||
<li><a href="#possible-improvements">Possible improvements</a></li>
|
||
<li><a href="#examples-of-use">Examples of use</a><ul>
|
||
<li><a href="#simple-command-line-usage">Simple command line usage</a></li>
|
||
<li><a href="#a-simple-demo-script">A simple demo script</a></li>
|
||
</ul></li>
|
||
</ul></li>
|
||
<li><a href="#links">Links</a></li>
|
||
</ul>
|
||
</nav>
|
||
</header>
|
||
<h2 id="overview">Overview</h2>
|
||
<p>This is the fourth show about the Bash functions I use, and it may be the last unless I come up with something else that I think might be of general interest.</p>
|
||
<p>There is only one function to look at this time, but it’s fairly complex so needs an entire episode devoted to it.</p>
|
||
<p>As before it would be interesting to receive feedback on this function and would be great if other Bash users contributed ideas of their own.</p>
|
||
<h2 id="the-range_parse-function">The <code>range_parse</code> function</h2>
|
||
<p>The purpose of this function is to read a string containing a range or ranges of numbers and turn it into the actual numbers intended. For example, a range like <code>1-3</code> means the numbers 1, 2 and 3.</p>
|
||
<p>I use this a lot. It’s really helpful when writing a script to select from a list. The script can show the list with a number against each item, then ask the script user to select which items they want to be deleted, or moved or whatever.</p>
|
||
<p>For example, I manage the podcasts I am listening to this way. I usually have two or three players with playlists on them. When the battery on one needs charging I can pick up another and continue listening to whatever is on there. I have a script that knows which playlists are on which player, and it asks me which episode I am listening to by listing all the playlists. I answer with a range. Another script then asks which of the episodes that I was listening to have finished. It then deletes the episodes I have heard.</p>
|
||
<p>Parsing a collection of ranges then is not particularly difficult, even in Bash, though dealing with some of the potential problems complicates matters a bit.</p>
|
||
<p>The function <code>range_parse</code> takes three arguments:</p>
|
||
<ol type="1">
|
||
<li>The maximum value allowed in the range (the minimum is fixed at 1)</li>
|
||
<li>The string containing the range expression itself</li>
|
||
<li>The name of the variable to receive the result</li>
|
||
</ol>
|
||
<p>An example of using the function might be:</p>
|
||
<pre><code>$ source range_parse.sh
|
||
$ range_parse 10 '1-4,7,3,7' parsed
|
||
$ echo $parsed
|
||
1 2 3 4 7</code></pre>
|
||
<p>The function has dealt with the repetition of 7 and the fact that the 3 is already in the range 1-4 and has sorted the result as a string that can be placed in an array or used in a <code>for</code> loop.</p>
|
||
<h3 id="algorithm">Algorithm</h3>
|
||
<p>The method used for processing the range presented to the function is fairly simple:</p>
|
||
<ol type="1">
|
||
<li>The range string is stripped of spaces</li>
|
||
<li>It is checked to ensure that the characters it contains are digits, commas and hyphens. If not then the function ends with an error</li>
|
||
<li>The comma-separated elements are selected one by one
|
||
<ul>
|
||
<li>Elements consisting of groups of digits (i.e. numbers) are stored away for later</li>
|
||
<li>If the element contains a hyphen then it is checked to ensure it consists of two groups of digits separated by the hyphen, and it is split up and the range of numbers between its start and end is determined</li>
|
||
<li>The results of the step-by-step checking of elements is accumulated for the next stage</li>
|
||
</ul></li>
|
||
<li>The accumulated elements are checked to ensure they are each in range. Any that are not are rejected and an error message produced showing what was rejected.</li>
|
||
<li>Finally all of the acceptable items are sorted and any duplicates removed and returned as a list in a string. If any errors occurred in the analysis of the range the function returns a ‘false’ value to the caller, otherwise ‘true’ is returned. This allows it to be used where a true/false value is expected, such as in an <code>if</code> statement, if desired.</li>
|
||
</ol>
|
||
<h3 id="analysis-of-function">Analysis of function</h3>
|
||
<p>Here is the function itself, which may be downloaded from the HPR website as <a href="http://hackerpublicradio.org/eps/hpr2483/range_parse.sh"><code>range_parse.sh</code></a>:</p>
|
||
<div class="sourceCode"><table class="sourceCode bash numberLines"><tr class="sourceCode"><td class="lineNumbers"><pre>1
|
||
2
|
||
3
|
||
4
|
||
5
|
||
6
|
||
7
|
||
8
|
||
9
|
||
10
|
||
11
|
||
12
|
||
13
|
||
14
|
||
15
|
||
16
|
||
17
|
||
18
|
||
19
|
||
20
|
||
21
|
||
22
|
||
23
|
||
24
|
||
25
|
||
26
|
||
27
|
||
28
|
||
29
|
||
30
|
||
31
|
||
32
|
||
33
|
||
34
|
||
35
|
||
36
|
||
37
|
||
38
|
||
39
|
||
40
|
||
41
|
||
42
|
||
43
|
||
44
|
||
45
|
||
46
|
||
47
|
||
48
|
||
49
|
||
50
|
||
51
|
||
52
|
||
53
|
||
54
|
||
55
|
||
56
|
||
57
|
||
58
|
||
59
|
||
60
|
||
61
|
||
62
|
||
63
|
||
64
|
||
65
|
||
66
|
||
67
|
||
68
|
||
69
|
||
70
|
||
71
|
||
72
|
||
73
|
||
74
|
||
75
|
||
76
|
||
77
|
||
78
|
||
79
|
||
80
|
||
81
|
||
82
|
||
83
|
||
84
|
||
85
|
||
86
|
||
87
|
||
88
|
||
89
|
||
90
|
||
91
|
||
92
|
||
93
|
||
94
|
||
95
|
||
96
|
||
97
|
||
98
|
||
99
|
||
100
|
||
101
|
||
102
|
||
103
|
||
104
|
||
105
|
||
106
|
||
</pre></td><td class="sourceCode"><pre><code class="sourceCode bash"><span class="co">#=== FUNCTION ================================================================</span>
|
||
<span class="co"># NAME: range_parse</span>
|
||
<span class="co"># DESCRIPTION: Parse a comma-separated list of numbers and "number-number"</span>
|
||
<span class="co"># ranges such as '1,3,5-7,9'</span>
|
||
<span class="co"># PARAMETERS: 1 - maximum limit of the range</span>
|
||
<span class="co"># 2 - entered range expression (e.g. 1-3,7,14)</span>
|
||
<span class="co"># 3 - name of the variable to receive the result</span>
|
||
<span class="co"># RETURNS: Writes a list of values to the nominated variable and returns</span>
|
||
<span class="co"># 0 (true) if the range parsed, and 1 (false) if not</span>
|
||
<span class="co">#===============================================================================</span>
|
||
<span class="kw">function</span><span class="fu"> range_parse</span> <span class="kw">{</span>
|
||
<span class="bu">local</span> <span class="va">max=${1?</span>range_parse: arg1 missing<span class="va">}</span>
|
||
<span class="bu">local</span> <span class="va">range=${2?</span>range_parse: arg2 missing<span class="va">}</span>
|
||
<span class="bu">local</span> -n <span class="va">result=${3?</span>range_parse: arg3 missing<span class="va">}</span>
|
||
|
||
<span class="bu">local</span> <span class="va">item</span> <span class="va">selection</span> <span class="va">sel</span> <span class="va">err</span> <span class="va">msg</span> <span class="va">exitcode=</span>0
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Remove spaces from the range</span>
|
||
<span class="co">#</span>
|
||
<span class="va">range=${range//</span> <span class="va">/}</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Check for invalid characters</span>
|
||
<span class="co">#</span>
|
||
<span class="kw">if [[</span> <span class="va">$range</span> =~ [^0-9,-]<span class="kw"> ]]</span>; <span class="kw">then</span>
|
||
<span class="bu">echo</span> <span class="st">"Invalid range: </span><span class="va">$range</span><span class="st">"</span>
|
||
<span class="bu">return</span> 1
|
||
<span class="kw">fi</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Slice up the sub-ranges separated by commas and turn all n-m expressions</span>
|
||
<span class="co"># into the intermediate values. Trim the trailing space from the</span>
|
||
<span class="co"># concatenation.</span>
|
||
<span class="co">#</span>
|
||
<span class="kw">until [[</span> <span class="ot">-z</span> <span class="va">$range</span><span class="kw"> ]]</span>; <span class="kw">do</span>
|
||
<span class="co">#</span>
|
||
<span class="co"># Get a comma-separated item</span>
|
||
<span class="co">#</span>
|
||
<span class="kw">if [[</span> <span class="va">$range</span> =~ [,]<span class="kw"> ]]</span>; <span class="kw">then</span>
|
||
<span class="va">item=${range%%</span>,*<span class="va">}</span>
|
||
<span class="va">range=${range#</span>*,<span class="va">}</span>
|
||
<span class="kw">else</span>
|
||
<span class="va">item=$range</span>
|
||
<span class="va">range=</span>
|
||
<span class="kw">fi</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Look for a 'number-number' expression</span>
|
||
<span class="co">#</span>
|
||
<span class="kw">if [[</span> <span class="va">$item</span> =~ [-]<span class="kw"> ]]</span>; <span class="kw">then</span>
|
||
<span class="kw">if [[</span> <span class="va">$item</span> =~ ^([0-9]<span class="dt">{1,})-([0-9]{1,}</span>)$<span class="kw"> ]]</span>; <span class="kw">then</span>
|
||
<span class="va">item=$(</span><span class="bu">eval</span> <span class="st">"echo {</span><span class="va">${item/</span>-<span class="va">/</span>..<span class="va">}</span><span class="st">}"</span><span class="va">)</span>
|
||
<span class="kw">else</span>
|
||
<span class="bu">echo</span> <span class="st">"Invalid sequence: </span><span class="va">${item}</span><span class="st">"</span>
|
||
<span class="va">item=</span>
|
||
<span class="va">exitcode=</span>1
|
||
<span class="kw">fi</span>
|
||
<span class="kw">fi</span>
|
||
<span class="va">selection+=</span><span class="st">"</span><span class="va">$item</span><span class="st"> "</span>
|
||
<span class="kw">done</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Check for out of bounds problems, sort the values and and make unique</span>
|
||
<span class="co">#</span>
|
||
<span class="kw">if [[</span> <span class="ot">-n</span> <span class="va">$selection</span><span class="kw"> ]]</span>; <span class="kw">then</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Validate the resulting range</span>
|
||
<span class="co">#</span>
|
||
<span class="kw">for</span> <span class="ex">i</span> in <span class="va">$selection</span><span class="kw">;</span> <span class="kw">do</span>
|
||
<span class="kw">if [[</span> <span class="va">$i</span> <span class="ot">-lt</span> 1 || <span class="va">$i</span> <span class="ot">-gt</span> <span class="va">$max</span><span class="kw"> ]]</span>; <span class="kw">then</span>
|
||
<span class="va">err+=</span><span class="st">"</span><span class="va">$i</span><span class="st"> "</span>
|
||
<span class="kw">else</span>
|
||
<span class="va">sel+=</span><span class="st">"</span><span class="va">$i</span><span class="st"> "</span>
|
||
<span class="kw">fi</span>
|
||
<span class="kw">done</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Report any out of range errors</span>
|
||
<span class="co">#</span>
|
||
<span class="kw">if [[</span> <span class="va">${err+</span><span class="st">"</span><span class="va">${err}</span><span class="st">"</span><span class="va">}</span><span class="kw"> ]]</span>; <span class="kw">then</span>
|
||
<span class="va">msg=</span><span class="st">"</span><span class="va">$(</span><span class="kw">for</span> <span class="ex">i</span> in <span class="va">${err}</span><span class="kw">;</span> <span class="kw">do</span> <span class="bu">echo</span> <span class="st">"</span><span class="va">$i</span><span class="st">"</span><span class="kw">;</span> <span class="kw">done</span> <span class="kw">|</span> <span class="fu">sort</span> -un<span class="va">)</span><span class="st">"</span>
|
||
<span class="va">msg=</span><span class="st">"</span><span class="va">${msg//</span><span class="st">$'</span><span class="dt">\n</span><span class="st">'</span><span class="va">/</span> <span class="va">}</span><span class="st">"</span>
|
||
<span class="bu">printf</span> <span class="st">"Value(s) out of range: %s\n"</span> <span class="st">"</span><span class="va">${msg}</span><span class="st">"</span>
|
||
<span class="va">exitcode=</span>1
|
||
<span class="kw">fi</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Rebuild the selection after having removed errors</span>
|
||
<span class="co">#</span>
|
||
<span class="va">selection=</span>
|
||
<span class="kw">if [[</span> <span class="va">${sel+</span><span class="st">"</span><span class="va">${sel}</span><span class="st">"</span><span class="va">}</span><span class="kw"> ]]</span>; <span class="kw">then</span>
|
||
<span class="va">selection=</span><span class="st">"</span><span class="va">$(</span><span class="kw">for</span> <span class="ex">i</span> in <span class="va">${sel}</span><span class="kw">;</span> <span class="kw">do</span> <span class="bu">echo</span> <span class="st">"</span><span class="va">$i</span><span class="st">"</span><span class="kw">;</span> <span class="kw">done</span> <span class="kw">|</span> <span class="fu">sort</span> -un<span class="va">)</span><span class="st">"</span>
|
||
<span class="va">selection=</span><span class="st">"</span><span class="va">${selection//</span><span class="st">$'</span><span class="dt">\n</span><span class="st">'</span><span class="va">/</span> <span class="va">}</span><span class="st">"</span>
|
||
<span class="kw">fi</span>
|
||
<span class="kw">fi</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Return the result</span>
|
||
<span class="co">#</span>
|
||
<span class="va">result=</span><span class="st">"</span><span class="va">$selection</span><span class="st">"</span>
|
||
|
||
<span class="bu">return</span> <span class="va">$exitcode</span>
|
||
<span class="kw">}</span>
|
||
</code></pre></td></tr></table></div>
|
||
<ul>
|
||
<li><p><b>Line 11</b>: There are two ways of declaring a function in Bash. The function name may be followed by a pair of parentheses and then the body of the function (usually enclosed in curly braces). Alternatively the word <code>function</code> is followed by the function name, optional parentheses and the function body. There is no significant difference between the two methods.</p></li>
|
||
<li><p><b>Lines 12 and 13</b>: The first two arguments for the function are stored in local variables <code>max</code> (the maximum permitted number in the range) and <code>range</code> (the string holding the range expression to parse). In both cases we use the parameter expansion feature which halts the script with an error message if these arguments are not supplied.</p></li>
|
||
<li><p><b>Line 14</b>: Here <code>local -n</code> is used for the local variable <code>result</code> which is to hold the name of a variable external to the function which will receive the result of parsing the expression. Using the <code>-n</code> option makes it a <em>nameref</em>; a reference to another variable. The definition in the Bash manual is as follows:</p></li>
|
||
</ul>
|
||
<blockquote>
|
||
<p><em>Whenever the nameref variable is referenced, assigned to, unset, or has its attributes modified (other than using or changing the nameref attribute itself), the operation is actually performed on the variable specified by the nameref variable’s value. A nameref is commonly used within shell functions to refer to a variable whose name is passed as an argument to the function.</em><br />
|
||
<br />
|
||
There is more to talk about with <em>nameref</em> variables, but we will leave that for another time.</p>
|
||
</blockquote>
|
||
<ul>
|
||
<li><p><b>Line 16</b>: Some other variables local to the function are declared here, and one (<code>exitcode</code>) is given an initial value.</p></li>
|
||
<li><p><b>Line 21</b>: Here all spaces are being removed from the range list in variable <code>range</code>.</p></li>
|
||
<li><p><b>Lines 26 to 29</b>: In this test the <code>range</code> variable is being checked against a regular expression consisting only of the digits 0-9, a comma and a hyphen. These are the only characters allowed in the range list. If the match fails an error message is written and the function returns with a ‘false’ value.</p></li>
|
||
<li><p><b>Lines 36-61</b>: This is the loop which chops up the range list into its component parts. Each time it iterates a comma-separated element is removed from the <code>range</code> variable, which grows shorter, and the test:</p>
|
||
<pre><code>until [[ -z $range ]]</code></pre>
|
||
will become true when nothing is left.
|
||
<ul>
|
||
<li><b>Lines 40-46</b>: This <code>if</code> statement looks to see if the <code>range</code> variable contains a comma, using a regular expression.
|
||
<ul>
|
||
<li>If it does a variable called <code>item</code> is filled with the characters of <code>range</code> up to the first comma. Then <code>range</code> is set to its previous contents without the part up to the first comma.</li>
|
||
<li>If there was no comma then <code>item</code> is set to the entirety of <code>range</code> and <code>range</code> is emptied. This is because this must be the last (or only) element.</li>
|
||
</ul></li>
|
||
<li><b>Lines 51-59</b>: At this point the element in <code>item</code> is either a plain number or a range expression of the form ‘<code>number-number</code>’. This pair of nested <code>if</code> statements determine if it is the latter and attempt to expand the range. The outer <code>if</code> tests <code>item</code> against a regular expression consisting of a hyphen, and if the result is true the inner <code>if</code> is invoked<a href="#fn1" class="footnoteRef" id="fnref1"><sup>1</sup></a>.
|
||
<ul>
|
||
<li><b>Line 52</b>: compares the contents of <code>item</code> against a more complex regular expression. This one looks for one or more digits, a hyphen, and one or more digits.
|
||
<ul>
|
||
<li>If found then <code>item</code> is edited to replace the hyphen by a pair of dots. This is inside braces as the argument to an <code>echo</code> statement. So, given <code>1-5</code> in <code>item</code> the <code>echo</code> will be given <code>{1..5}</code>, a brace expansion expression. The <code>echo</code> is the command of an <code>eval</code> statement (needed to actually execute the expansion), and this is inside a command expansion. The result should be that <code>item</code> is filled with the numbers from the expansion so <code>1-5</code> becomes ‘<code>1 2 3 4 5</code>’!</li>
|
||
<li>If the regular expression does not match then this is not a valid range, so this is reported in the <code>else</code> branch and <code>item</code> is cleared of its contents. Also, since we want this error reported to the caller we set <code>exitcode</code> to 1 for later use.</li>
|
||
</ul></li>
|
||
<li><b>Line 60</b>: Here a variable called <code>selection</code> is being used to accumulate the successive contents of <code>item</code> on each iteration. We use the <code>+=</code> form of assignment to make it easier to do this accumulation. Notice that a trailing space is added to ensure none of the numbers collide with one another in the string.</li>
|
||
</ul></li>
|
||
</ul></li>
|
||
</ul>
|
||
<!-- -->
|
||
<ul>
|
||
<li><b>Lines 66-97</b>: This is an <code>if</code> statement which tests to see if the variable <code>selection</code> contains anything. If it does then the contents are validated.
|
||
<ul>
|
||
<li><b>Lines 71-77</b>: This is a loop which cycles through the numbers in the variable. It is a feature of this form of the <code>for</code> loop that it operates on a list of space-separated items, and that’s what <code>selection</code> contains.
|
||
<ul>
|
||
<li><b>Lines 72-76</b>: This <code>if</code> statement checks each number to ensure that it is in range between 1 and the value in the variable <code>max</code>.
|
||
<ul>
|
||
<li>If it is not in range then the number is appended to the variable <code>err</code></li>
|
||
<li>If it is in range it is appended to the variable <code>sel</code></li>
|
||
</ul></li>
|
||
</ul></li>
|
||
<li><b>Lines 82-87</b>: This <code>if</code> statement tests to determine whether there is anything in the <code>err</code> variable. If it contains anything then there have been one or more errors, so we want to report this. The test used here seems very strange. The reason for it is discussed below in the <a href="#explanations">Explanations</a> section, explanation 1.
|
||
<ul>
|
||
<li><b>Line 83</b>: The variable <code>msg</code> is filled with the list of errors. This is done with a command substitution expression where a <code>for</code> loop is used to list the numbers in <code>err</code> using an <code>echo</code> command and these are piped to the <code>sort</code> command. The <code>sort</code> command makes what it receives unique and sorts the lines numerically. This rather involved pipeline is needed because <code>sort</code> requires a series of lines, and these are provided by the <code>echo</code>. This deals with the possible duplication of the errors and the fact that they are not necessarily in any particular order.</li>
|
||
<li><b>Line 84</b>: Because the process of sorting the erroneous numbers and making them unique has added newlines to them all we use this statement to remove them. This is an example of parameter expansion, and in this one the entire string is scanned for a pattern and each one is replaced by a space. There is a problem with replacing newlines in a string however, since there is no simple way to represent them. Here we use <code>$'\n'</code> to do this. See the <a href="#explanations">Explanations</a> section below (explanation 2) for further details.</li>
|
||
<li><b>Line 85 and 86</b>: The string of erroneous number is printed here and <code>exitcode</code> is set to 1 so the function can flag that there has been an error when it exits. It doesn’t exit though since some uses will simply ignore the returned value and carry on regardless.</li>
|
||
</ul></li>
|
||
<li><b>Lines 92-96</b>: At this point we have extracted all the valid numbers and stored them in <code>sel</code> and we want to sort them and make them unique as we did with <code>err</code> before returning the result to the caller. We start by emptying the variable <code>selection</code> in anticipation.
|
||
<ul>
|
||
<li><b>Line 93</b>: This <code>if</code> statement checks that the <code>sel</code> variable actually contains anything. This test uses the unusual construct <code>${sel+"${sel}"}</code>, which was explained for an earlier test. (See explanation 1 in the <a href="#explanations">Explanations</a> section below).</li>
|
||
<li><b>Line 94 and 95</b>: These rebuild <code>selection</code> by extracting the numbers from <code>sel</code>, sorting them and making them unique, and then removing the newlines this process has added. See the notes for lines 82-87 above and explanation 2 below.</li>
|
||
</ul></li>
|
||
</ul></li>
|
||
<li><p><b>Line 102</b>: Here the variable <code>result</code> is set to the contents of <code>selection</code>. Now, since <code>result</code> is a <em>nameref</em> variable containing the name of a variable passed in when the <code>range_parse</code> function was called it is that variable that receives the result.</p></li>
|
||
<li><p><b>Line 104</b>: Here the function returns to the caller. The value returned is whatever is in <code>exitcode</code>. By default this is zero, but if any sort of error has occurred it will have been set to 1, as discussed earlier.</p></li>
|
||
</ul>
|
||
<h3 id="explanations">Explanations</h3>
|
||
<ol type="1">
|
||
<li><p>The expression <code>${err+"${err}"}</code> (see Lines 82-87 above), also <code>${sel+"${sel}"}</code> (see Line 93 above): As far as I can determine this strange expression is needed because of a bug in the version of Bash I am running.<br />
|
||
<br />
|
||
In all of my scripts I include the line <code>set -o nounset</code> (<code>set +u</code> is equivalent) which has the result of treating the use of unset variables in parameter expansion as a fatal error. The trouble is that either <code>err</code> and <code>sel</code> might be unset in this function in some circumstances. This will result in the function stopping with an error. It should be possible to test a variable to see whether it is unset without the function crashing!<br />
|
||
<br />
|
||
This expression is a case of a parameter expansion of the <em><code>${parameter:+word}</code></em> type, but without the colon. It returns a null string if the parameter is unset or null or the contents if it has any - and it does so without triggering the <em>unset variable</em> alarm.<br />
|
||
<br />
|
||
I don’t like resorting to <em>“magic”</em> solutions like this but it seems to be a viable way of avoiding this issue.</p></li>
|
||
<li><p>The expression <code>$'\n'</code> (see Line 84 above): This is an example of ANSI-C quoting. See the GNU Bash Reference Manual in the <a href="https://www.gnu.org/software/bash/manual/bash.html#ANSI_002dC-Quoting" title="ANSI-C quoting in Bash">ANSI-C Quoting</a> section for the full details.<br />
|
||
<br />
|
||
The construct must be written as <code>$'string'</code> which is expanded to whatever characters are in the <em>string</em> with certain backslash sequences being replaced according to the ANSI-C standard. This allows characters such as newline (<code>\n</code>) and carriage return (<code>\r</code>) as well as Unicode characters to be easily inserted. For example <code>echo $'\U2192'</code> produces → (in a browser and in many terminals).</p></li>
|
||
</ol>
|
||
<h3 id="possible-improvements">Possible improvements</h3>
|
||
<p>This function has been around the block for quite a few years. I wrote it originally for a script I developed at work in the 2000’s and have been refining and using it in many other projects since. Preparing it for this episode has resulted in some further refinements!</p>
|
||
<ul>
|
||
<li><p>The initial space removal means that <code>'7,1-5'</code> and <code>'7 , 1 - 5 '</code> are identical as far as the algorithm is concerned. It also means that <code>'4 2'</code>, which might have been written that way because a comma was omitted, is treated as <code>'42'</code> which might be a problem.</p></li>
|
||
<li><p>The command substitutions which sort lists of numbers and make them unique have to make use of the <code>sort</code> command. Ideally I’d like to avoid using external programs in my Bash scripts, but trying to do this type of thing in Bash where <code>sort</code> does a fine job seems a little extreme!</p></li>
|
||
<li><p>The reporting of all of the numbers which are out of range could lead to a slightly bizarre error report if called with arguments such as <code>20 '5-200'</code> (where the second zero was added in error). Everything from 21-200 will be reported as an error! The function could be cleverer in this regard.</p></li>
|
||
</ul>
|
||
<h3 id="examples-of-use">Examples of use</h3>
|
||
<h4 id="simple-command-line-usage">Simple command line usage</h4>
|
||
<pre><code>$ source range_parse.sh
|
||
$ range_parse 10 '1-3,9,7' mylist
|
||
$ echo "$mylist"
|
||
1 2 3 7 9
|
||
|
||
$ range_parse 10 '9-6,1,11' mylist
|
||
Value(s) out of range: 11
|
||
$ echo "$mylist"
|
||
1 6 7 8 9
|
||
|
||
$ range_parse 10 1,,2 somevar
|
||
$ echo "$somevar"
|
||
1 2</code></pre>
|
||
<p>The <code>range_parse</code> function does not care what order the numbers and ranges are organised in the comma-separated list. It does not care about range overlaps either, nor does it care about empty items in the list. It flags items which are out of range but still prepares a final list.</p>
|
||
<h4 id="a-simple-demo-script">A simple demo script</h4>
|
||
<p>The simple script called <code>range_demo.sh</code>, which may be <a href="http://hackerpublicradio.org/eps/hpr2483/range_demo.sh">downloaded</a> from the HPR website is as follows:</p>
|
||
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="co">#!/bin/bash -</span>
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Test script to run the range_parse function</span>
|
||
<span class="co">#</span>
|
||
|
||
<span class="kw">set</span> <span class="ex">-o</span> nounset # Treat unset variables as an error
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Source the function. In a real script you'd want to provide a path and check</span>
|
||
<span class="co"># the file is actually there.</span>
|
||
<span class="co">#</span>
|
||
<span class="bu">source</span> range_parse.sh
|
||
|
||
<span class="co">#</span>
|
||
<span class="co"># Call range_parse with the first two arguments provided to this script. Save</span>
|
||
<span class="co"># the output in the variable 'parsed'. The function is called in an 'if'</span>
|
||
<span class="co"># statement such that it takes different action depending on whether the</span>
|
||
<span class="co"># parsing was successful or not.</span>
|
||
<span class="co">#</span>
|
||
<span class="kw">if</span> <span class="ex">range_parse</span> <span class="st">"</span><span class="va">$1</span><span class="st">"</span> <span class="st">"</span><span class="va">$2</span><span class="st">"</span> parsed<span class="kw">;</span> <span class="kw">then</span>
|
||
<span class="bu">echo</span> <span class="st">"Success"</span>
|
||
<span class="bu">echo</span> <span class="st">"Parsed list: </span><span class="va">${parsed}</span><span class="st">"</span>
|
||
<span class="kw">else</span>
|
||
<span class="bu">echo</span> <span class="st">"Failure"</span>
|
||
<span class="kw">fi</span>
|
||
|
||
<span class="bu">exit</span></code></pre></div>
|
||
<p>An example call might be:</p>
|
||
<pre><code>$ ./range_demo.sh 10 1,9-7,2
|
||
Success
|
||
Parsed list: 1 2 7 8 9</code></pre>
|
||
<p>If you download these files and test the function and find any errors <em>please</em> let me know!!</p>
|
||
<h2 id="links">Links</h2>
|
||
<ul>
|
||
<li>The <a href="https://www.gnu.org/software/bash/manual/bash.html">GNU Bash Reference Manual</a>
|
||
<ul>
|
||
<li><a href="https://www.gnu.org/software/bash/manual/bash.html#ANSI_002dC-Quoting">ANSI-C Quoting</a> section</li>
|
||
</ul></li>
|
||
<li>Previous HPR episodes in this group <em>Useful Bash functions</em>:
|
||
<ul>
|
||
<li><a href="http://hackerpublicradio.org/eps.php?id=1757">Part 1</a>: ‘<code>pad</code>’ and ‘<code>yes_no</code>’ functions</li>
|
||
<li><a href="http://hackerpublicradio.org/eps.php?id=2096">Part 2</a>: ‘<code>yes_no</code>’ revisited</li>
|
||
<li><a href="http://hackerpublicradio.org/eps.php?id=2448">Part 3</a>: ‘<code>read_value</code>’, ‘<code>check_value</code>’ and ‘<code>read_and_check</code>’</li>
|
||
</ul></li>
|
||
<li>Download the <a href="http://hackerpublicradio.org/eps/hpr2483/range_parse.sh"><em>range_parse</em></a> function and the <a href="http://hackerpublicradio.org/eps/hpr2483/range_demo.sh"><em>range_demo.sh</em></a> test script.</li>
|
||
</ul>
|
||
<!--
|
||
vim: syntax=markdown:ts=8:sw=4:ai:et:tw=78:fo=tcqn:fdm=marker
|
||
-->
|
||
<section class="footnotes">
|
||
<hr />
|
||
<ol>
|
||
<li id="fn1"><p>Why do it this way? I did a double-take while preparing these notes wondering why I had organised the logic here in this way.<br />
|
||
<br />
|
||
The first part of the loop is concerned with getting the next item from a comma-separated list. At that point the contents of <code>$item</code> is either a bare number or a <code>'number-number'</code> range. The differentiator between the two is a hyphen, so checking for that character allows the complex regular expression on line 52 to be omitted if it is not there.<br />
|
||
<br />
|
||
If you can think of a better way of doing this please let me know in the comments or by email.<a href="#fnref1">↩</a></p></li>
|
||
</ol>
|
||
</section>
|
||
</article>
|
||
</main>
|
||
</div>
|
||
</body>
|
||
</html>
|