hpr-knowledge-base/hpr_transcripts/hpr2689.txt

Episode: 2689
Title: HPR2689: Bash Tips - 14
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr2689/hpr2689.mp3
Transcribed: 2025-10-19 07:35:33

---

This is HPR Episode 2689 entitled Bash Tips 14 and is part of the series Bash Cripting.
It is hosted by Dave Morris and is about 28 minutes long and carries an explicit flag.
The summary is more about loops for loop, break and continue.
This episode of HPR is brought to you by archive.org.
University Access to All Knowledge by heading over to archive.org forward slash donate.
Hello everybody. Welcome to Hacker Public Radio. This is Dave Morris. I'm doing a show
in my Bash Tips series today and I've recently done a bunch of shows entitled Making
Decisions in Bash and looked at loops there. I looked at while and until loops but I
mentioned that there's another loop construct which uses a four verb, four loops and I've
not really covered that. I've used them in many shows. That really fully explaining them
and I thought this was the time to make good that deficiency. While I'm at it I'm also
going to look at the command break and continue which are used for when using loops.
So in Bash there are two formats of the four command. The first one I'm actually using
the syntax that's used in the GNU Bash Manual which is less pretty but is probably more
correct than sort of thing I've been using in previous shows. So the actual command consists
of the word four FOR, then a space and then the name of a variable. Then that can be followed
by optionally the word in and then the list or words. If you are doing this, if you have
these then there's a semicolon that follows that or indeed a new line. Whenever there's
a semicolon in this syntax you can also be replaced by a new line so you can spread out
your command over multiple lines. Then the word do and then a list of commands of any
anything similar to the other loops who've looked at another semicolon or new line and
the final word is done. So that's the whole structure. The thing that's the in words part
refers to is either a literal list which I'll do an example of in a moment or something
which will expand to provide a list or whatever sort of expansion is relevant. What the loop
does is it cycles for each member of the list setting the variable whatever you called it
to each successive member and executing the commands at each iteration. So my example is
four color in red, green, blue. So color is the name of the variable and the list is red, green,
blue. They're just words unquoted. If the words needed had spaces in them, we wanted to have bright
red with a space in between. You'd have to quote them to make it just the one word. Similar sort of
issues pertain he has. You would find all over bash when you're using arguments to programs
or scripts or whatever. Then after that list is a semicolon and do. I like to put the two things
on the same line. You don't have to. Then the next line is echo and in double quotes dollar color
and then the last line is done. Then I like to indent the commands that are inside the loop just
to make them clearer. And in fact if I use the Vim Editor with bash or SH as its syntax file then
we'll do that automatically for me. So just like the look of it it's easy to read. So running that
will output three color names one per line. I'm bothered to run that one because it's so simple.
Another example might be four space file or files the variable space in. Then the list is not
actually a list but it's an expansion expression. So asterisk.mp3 semicolon space do. Now we'll in the
the current directory that you are running this. It will look for any files which end with
dot mp3. And one pitfall that I haven't gone into in the notes here but I mentioned in earlier
earlier shows in this group is that if there are no mp3 files in that directory you will get back
the string asterisk dot mp3 and you can control whether you get that or not with some options which
I went into in quite a lot of detail. So be aware of that as I was typing this I'd forgotten to be
perfectly honest. So I'm mentioning it now because it's a good chance you you might have a memory
like mine you've gotten to. Anyway the command in this loop is echo dollar file in quotes. So we'll
just print out the name of the file. So it's just a way of doing the equivalent of ls asterisk.mp3
except it's a bit cleaner you don't get all the other garbage you get with ls. So a list like that
might be empty though not in the case of sts.mp3 and as you've enabled the null glob option but it
might expand to nothing might be empty. So it's perfectly legal to have a format which consists of
for name whatever the name of the variable is in and then nothing then a semicolon. So that's
okay nothing will happen. It won't run at all because there's nothing to do. Now there's a special
form of the this format of the loop where you can write it without the in words part which case
you don't actually need to put a semicolon before the do. What this form does is it cycles through all
the positional parameters. So when you start your bash shell up you usually don't have positional
parameters attached to you but if you create a script and run that then you can as you know hand
it some variables strings or numbers or whatever and those are the arguments and they're called
dollar one dollar two dollar three etc. So running it this way is the equivalent to writing
four space names base in and then in double quotes dollar at sign. Dollar at sign is one of the
special variables within bash which refers to the entire list of arguments that you have in a
script or a function. So I've written a pretty straightforward example which is called bash 14
EX1 and you can download it if you want to but it's also here to be looked at and it simply consists
of for loop for arg space do called a variable arg because it's an argument echo in double quotes
dollar arg done. So if you run that and give it the four words let joy be unconfined as you would
do because then you get back one per line let joy be unconfined. The return states of four command
is the exit status of the last command that executes within it so that list of commands the last one
whatever it returns to a fourth will or indeed something which is not zero or one will be what the
for loop returns. I personally never found that would be particularly useful but you might want to
test that sort of thing I'm not sure. If there are no items when the list word becomes to be expanded
no commands are executed and the return status of the whole shebang is zero she's pretty long. So
that's the first format you've seen me using it in various examples and so on. There'll be other
cases in this this collection of scripts where I use variants of it. So the second format is a bit
more complicated it consists of the word four then a space then three expressions enclosed in double
parentheses then the semicolon do the list of command semicolon done. So I've used the the usual
syntax here and the expressions within the double parentheses I'm referring to as expression one
expression two expression three separated by semicolon so you have to use this this particular
format. So these are arithmetic expressions as I think I just said and it uses these the loop
uses these to determine how many times to to iterate. So expression one is an arithmetic expression
which is evaluated at the start and it usually consists of a variable being set to some value
where that value can be an expression or whatever. There needs to be a variable because the loop
is tracking the iterations by using the variable that's not strictly true but if you can if you
will come on to why I hesitated there slightly in a moment essentially it's usual to to set a
variable to a value. There'll be examples shortly. Expression two is also an arithmetic expression
which is evaluated at each iteration of the loop. When each time it evaluates to a non-zero value
the commands in the loop are executed. Expression three is another arithmetic expression which is
evaluated each time expression two evaluates to a non-zero value so traditionally that's used to
increment or decrement your variable. So here's an example. So we've got four space two open
parentheses i equals one so the variable i is being set to one. Remember the double parentheses
means it's an arithmetic expression. We'll list up in this case and the all the rules about
spaces and stuff have gone out the window you can use them or not use them as you wish. So after
i equals one semicolon then the second expression is i less than 10 so that the comparison is
going to be run every time the loop iterates to see if the variable i is less than 10 and the
third expression after semicolon is i plus plus so that's one of those arithmetic increment
that's actually a posting increment so it will return i with one added to it and then after
the closed double parentheses we've got semicolon space to do and then in the body of the loop
that one command we have is echo dollar i. So that will just output the numbers one to nine one
per line. Now as I alluded to this quite a lot of flexibility allowed in the expressions you can
use in this format and the rules of shell arithmetic apply and I've put a reference to shell
arithmetic but we've talked about this in recent shows in fact so it's all very flexible about
spaces you don't need them after the open parentheses and nor before the closed parentheses could be
nice to add them I personally don't but I've definitely put them between the expressions and
that I also put them either sides of an equal sign when setting up the variable which you can't
normally do in bash. The paths I want won't deal with that which I've always felt was the same.
When you're referring to variables you don't need the leading dollar in fact if you put it in
it does have some side effect I haven't gone into what they are here I'm not sure I need to
particularly but I'm going to do a show after this one about potential traps that you might fall
into when working with loops so if I come across any any particular problems in that area I might
go into a bit of detail there but for the moment I don't think it's a big deal you should try not
to use the dollar sign in front of variables in this case you don't need them and of course within
these expressions any of the shell arithmetic arithmetic operators can be used so and there's a lot
of them as we saw I think one of the the recent shows there's a huge big list and I've referred to
the page I got that list from so you can go and look at that so my next example I'm setting i2
and then an expression of 78 plus 20 over 2 minus 4 times 12 which surprise surprise the answer
to that is 1 but it's just me being silly and giving you a more complex expression and then instead
of i being less than 10 I've put i not equal to 10 we should achieve the same thing
so that will just do exactly the same things the previous simpler looking like but it just shows
how much flexibility there is I thought I'd do a more complex example of what you can do in one
of these these types of for loops one of the arithmetic operators that you could use if you
wished is the comma operator and there's not a lot of information about it if you look in the
GNU bash manual at the point I've I've referred to the one in the list there's simply a thing that
says expression 1 comma expression 2 and the notation to it says comma which is yeah there's
there's very very little documentation about it I think possibly because when they added this to
bash it was added possibly at the request of c programmers because it's a funk it's a it's a
facility that's available in c basically it's a list of expressions that get executed and the
result of the whole list is the result the last expression so preamble to what I'm actually doing
here my for loop consists of four space open double parentheses i equals 1 right big deal r but then
there's a comma and then I've got j equals 100 semicolon so in that first expression there are
there's a comma comma list the list can actually be multiple but I've just made two entries here
you can have as many as you want to think I'm not sure there's a limit I haven't
experimented with that the expression to the test bit is checking to see whether i is less than
or equal to 10 and then the third expression increments i by 1 so it's i plus plus but there's
another comma and we've got j plus equals 10 so the variable j is being incremented at each time
around the loop by 10 so the body of the loop consists of an echo where we're going dollar i and
dollar j and there's done I've written this out as if it was being typed on the command line in fact
it was just to so I could catch it and put it in the notes and when it runs you get the values
1 to 10 for i and you get 100 incrementing in steps of 10 out to 194 j so you can see that would
that that could be really quite powerful you could do some clever things with that so there's a
little bit of stuff in the notes but experience a bit more than i've done there this one's got a
downloadable copy which is bash14ex2.sh you can grab and play with if you find it interesting or
want to experiment to see what else you can do with it so the final point about this this format
the full loop is that if any of the expression 1 2 and 3 is missing then it's taken by the
loop to be 1 so if you wrote 4 open double parenthesis i put spaces here just to make it clear
a space semicolon space semicolon space but you could just type two semicolons one after the other
in other words the the three expressions are missing completely and the closed double parenthesis
comma do and then the commands semicolon done however else you prefer to write it out
and this is an infinite loop because everything returns 1 so it'll always have a non-zero value
in the test so we'll just keep running forever not incrementing anything or doing anything
there's no variable to be incremented there's nothing no variable it could be used in the
body of the full loop it will just run forever so the return value of this type of four command
with the arithmetic expressions is not surprisingly the exit stages of the last command in the
command list but it can also return the value false if any of the expressions is invalid so if
you somehow manage to divide by zero or something like that it will return false value and stop
didn't actually test that i should maybe have done i'll leave that for you to look at so the other
topic i wanted to cover i mentioned is the break and continue command and we've we've seen these
i think in various contexts through the the other shows under the bash tips heading um
definitely use continue in the recent past didn't explain it all in all that much detail
and to try and explain it a little bit more this time both of these commands are built in one so
they're not exterior programs anything they're part of bash they're in fact inherited from
bash's predecessor the born shell and i've given the link to to them in the canoe bash manual
they're both used for changing the sequence of execution of a loop that's a full loop a while loop
and tin or an until loop when you to select loop which is effectively a loop yeah we haven't looked
at that we need to do so and i haven't got it scheduled up we will look at it at some point
because it's actually quite useful and we need to see how it can be used so let's look at the break
command the break command is the word break followed optionally by an integer which is great
than or equal to if you emit this integer then it's taken to be it specifies that the
nth enclosing loop is to be exited so my example is fairly simple it's a full loop using the
format with a list of words for i in and then i've used a double brace expansion the first brace
expansion is open curly bracket a dot dot c close curly bracket then another curly bracket
opening one dot dot three close curly brackets semicolon space do you'll remember and i've
referenced back to the show where i talked about this quite a lot of detail actually i did it
it's very cool i think it's a fun fun feature 1884 i talked about it and i've also referenced the
canoe bash manual chapter on brace expansion so that loop we haven't finished talking about what's
in it yet but the loop will generate the values a one a two a three b one b two b three etc
and it'll it'll do do that until it's used them all up so inside the loop we echo that that
variable i and but we've added another thing here which is a test so in the single square brackets
we put in quotes dollar i equals equals quotes b two close quotes close square brackets so we're
testing here to see if the the variable i contains the string b two this is a command list so it's
followed by double ampers and and then the word break so if we if we find get back a value of b two
then the loop just stops it exits when you see it running and i've run it and captured the output
you get a one a two a three b one b two and then it stopped so here's a more complex example
this one contains a loop within a loop the inner loop simply repeats the current value of the
variable i three times and it uses echo space minus n to suppress the new line so you just get
them on the same line this case we want to exit both loops whenever i get to b two i want to stop
both loops right the first four is the same as the one we saw in the previous example then there's
one inside it four j in then in braces one dot dot three and then we echo minus n in quotes dollar
i then the test inside this inner loop is dollar i equals equals b two that these are quoted variable
and string and ampersand ampersand this time I've got a brace around two two commands one is echo
just because after the last thing it was printed no new line was was output so we want to
output a new line this time we're using break space two and there's semicolons between these we
talked about this stuff in an earlier show how the the curly brackets are actually effectively
commands they have to be separated out and the things within them have to be separated out by new
lines or semicolons so that next is the done which closes the j loop then we have an echo
so that every when the thing is running before it's killed by the break then we output a new line
after the three repeats of the value of i hopefully that makes sense and that's an example what it
look like a one a one a one a two a two a two etc when you get to b two just get the one and then the
the test detects that it is the required value and it breaks both loops it kills the inner loop
and the outer loop the whole shebang stops i put these loops that i've just been using as examples
into a file that you can download if you want to to experiment with them and it's called bash 14
e x three dot s eight so that's what break does it exits loops and there's the continue command
and continue is that the word continue followed by a space and an optional number and it's
take it be one if it's missing and it's it's got to be one or greater the command's effect is it is
that the end enclosing loop must be resumed which is amusingly the bash manual terminology here
when it says be resumed what it actually means is wherever the continuous is invoked the rest of
that loop is skipped and then the loop continues with the next iteration that it would normally have
done so i've got an example here because this is getting a little more complicated not much
but just a little bit bash 14 e x four dot s h here we have pretty much the same examples we saw
before with the loop within a loop where a is being set to the values a one a two a three etc
the variable i is being set to these i should say then j within it is being set to values one two
three is and so on and we're outputting the value of i with a echo hyphen n but the test is
different this time using the extended test format with the double open and clothes brackets
square brackets and we're comparing dollar i with a glob style string match which is simplicance
is a b question mark so if the contents of a variable i begins with the b and contains another
letter another character then the match will trigger and we are doing similar things we did with the
with the break example issuing an echo command which just generates the new line that would be
missed otherwise and this time we're using continue we're using continue with the number two so
that will resume two loops probably makes more sense when you look at what it does it causes the
outer loop to continue with the next letter number combination but it would be using to set
the variable i to if you run it you get similar stuff to the previous example a one a one a one
etc and then when we get to the b's we only get b one once b two once b three once because each of
these matches the test so the continue command simply jumps past everything else within the nested
loops and goes on to the next value it's not spectacularly useful example but i'm sure that hopefully
explains that all of the stuff that's going on and will give you some thoughts about how you
would use it yourself whenever more specific requirement so i'm going to finish off with an example
this one's bash 14 EX5 it's not hugely complex but it's showing that the list when you're dealing with
the four loop in the first format with it using four variable in and list of words that list
can be anything so this time i've gone to my favorite user share dict words that file of huge numbers
of words i'm using the shuff command which is going to grab 10 random words from it but i've
used grab in this case to filter out all the words that end with apostrophe s because so many of
these seem to be arbitrary and ridiculous as i said in the notes at one point i got back the
word logger head apostrophe s well i can't imagine when you would use that usually use the logger
heads as at logger heads meaning in an aggressive confrontation so when would you ever
talk about the possessiveness of such a thing and if you're talking about logger head turtles you'd
put the possessive part on the turtle surely anyway i mean you could it's not that it's not
English but it just seems stupid so this one is just a very simple loop except that the source
of the words is a command substitution so we've got four w in then dollar open parenthesis we've
got the grep grep space hyphen capital E that invokes the er form of regular expressions
then space hyphen v which says give me all of the lines which don't match the regular expression
and the regular expression consists of in double quotes apostrophe s dollar so anything that ends
with apostrophe s the end of the line we don't want then the file we're dealing with the
is user shared dict words we pipe that into the shuff command with hyphen n space 10 so we get back
10 of the words that grep returns and then semicolon closed parenthesis semicolon do and we
simply echo the word dollar double and that's the end of the loop and it's done that ends it
and um i gave an example of running this and they're nice they're random you could do something
useful with them i've just listed them so please enjoy so that's the end that's what i'm going to
talk about you've been listening to hecka public radio at hecka public radio dot org we are a
community podcast network that release the shows every weekday Monday through Friday today show
like all our shows was contributed by an hbr listener like yourself if you ever thought of
recording a podcast then click on our contributing to find out how easy it really is hecka public
radio was founded by the digital dog pound and the infonomicon computer club and it's part of
the binary revolution at binwreff.com if you have comments on today's show please email the host
directly leave a comment on the website or record a follow-up episode yourself unless otherwise
stated today's show is released on the creative comments attribution share a light 3.0 license