hpr-knowledge-base/hpr_transcripts/hpr1903.txt

Episode: 1903
Title: HPR1903: Some further Bash tips
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr1903/hpr1903.mp3
Transcribed: 2025-10-18 10:59:54

---

This is HPR episode 1903 entitled Some Further Mash Tips and is part of the series Mash
Crypting.
It is hosted by Dave Morris and is about 29 minutes long.
The summary is some more information about types of expansion in Bash.
This episode of HPR is brought to you by AnanasThost.com.
Get 15% discount on all shared hosting with the offer code HPR15 that's HPR15.
Better web hosting that's honest and fair at AnanasThost.com.
Hello everyone, this is Dave Morris, got another talk about Bash for you today.
Now, what I've been doing so far with Bash is to, I suppose, just sort of pick and choose
bits and pieces that I've found myself and thought you might be interested in or things
that have taken me a long while to understand that I wanted to make sure I understood
and so I decided to share them with you.
But the way things have come together, I've been concentrating to a fair degree on the
subject of expansion.
That's where Bash processes the command line and does various things to it to prepare
it.
It's come under the general heading of expansion.
So we looked at Brace Expansion in the last one of these 1884.
We looked at Parameter and Variable Expansion, which was, I think, the first one I did in
this series, 1648, was the number.
Well, today we're going to look at Tilder Expansion and Command Substitution.
I've listed out the various types of expansion in the notes and there are long notes for
this, by the way, and it's, there's quite a few subjects there, so I'm just sort of doing
them slowly in chunks to avoid producing enormously long tedious shows.
So let's look at Tilder Expansion.
This is a convenient way of looking, referring to, I should say, a home directory when you're
writing a file path, when you're L.S.ing it or C.D.ing to it or something.
There are some other uses of Tilder Expansion as well, which I have to admit, I didn't
know about until I started putting this show together, so this is why these are fun to
do because I learned stuff along the way.
So if you use the Tilder all by itself, that's the first form, imagine you're logged
into your user account on a Linux machine and you've changed directory to the documents
directory.
I use that because you tend to get one of those whether you wanted it or not.
Now, if you wanted from there to look at your .bashrc file, you'd never look to that in
this series.
It's where you can set up lots of stuff about a bash.
I should add it to the list, I think, talk about it sometime.
Anyway, if you wanted to look at it, then you could type less, less being a useful way
of looking at stuff in a paged form, space.dot slash.bashrc.
The dot dot just means the directory above this one, the parent directory, so that would
do it.
You could also type less space.dot home, where home is in capitals slash.bashrc, so that
means look for the .bashrc in my home directory and show me that.
The third possibility, probably others as well, would be less space tilde slash.bashrc.
So what this does, what the tilde does is the same as what the dollar home example did.
It looks in that directory to find the .bashrc.
In fact, the tilde, the plane tilde like this is a direct equivalent.
It actually uses the contents of the home variable.
Now, when I was looking at this, I thought, I wonder what happens if you change the home variable
and of course it does work, if you change it and then cd to tilde or ls tilde, you see
a different directory and I put an example in the notes of, I just use echo followed
by a space in the tilde and that then just shows you what's generated by the tilde.
And I use a demo account for this and so I get back slash home slash hpr demo.
If I cd to documents again and type the command, it's a variable assignment, in fact, home
that's in capitals equals dollar pwd.
Now, pwd is a variable that contains the current directory you're in, it's changed
for you behind the scenes by bash.
So what I've done is to set home to this documents directory.
If I then do echo tilde, I see slash home slash hpr demo slash document.
So this is all fine and dandy but I then discovered this is enormously confusing
so I would not recommend that you do it but it's just a point really to explain what's
going on.
So as I said in the notes, this is a demonstration, not recommendation.
Now the next form we look at is where the tilde is followed by a login name or what some
people call a username and not too surprisingly, that refers to the home directory of that
login name.
So I put in my example, echo space tilde hpr demo, so that's the login name and it comes
back saying slash home slash hpr demo, not too surprising.
Then I typed echo space tilde postgres, that's the user that is the sort of controlling
them master user for the postgres, postgresql database system and it has the home directory
slash bar slash lib slash postgresql, why they have these director's, the home director's
isn't, is a subject for another day but that's what it does.
It then occurred to me when I was writing these notes that maybe you should be aware of
how to, if you ever did fiddle with your home variable, how you could set it back to something
sensible again, you can just log out and log by any again of course.
But if you did what I've done in my example here, which is home, that's in capitals equals
tilde hpr demo from, from my particular case, then what that will do is we just saw is it
will return the home directory of this user and it gets it from the et cetera password
file, et cetera password is the, is the place where a lot of stuff is to find about
user names, about login names, so echo tilde then gives me the proper home directory
for the account, another sort of slight warning, which may not be relevant to you, is when
you're typing the login name, if you've forgotten what it is exactly or had to spell it
and you know it begins with something or other, like many things in tab, you can, in bash,
you can use a tab comma, a tab key to get auto completion, well this works here too,
which is fine.
So if I were to type echo tilde hpr and then press tab, then there's only one hpr, only
one login name beginning with hpr on my machine, so it'd come back with hpr demo, but I used
to work in a university which had various points in the year up to about 50,000 login
names, all of which were available to the tilde command, so if you pressed a tab after
tugging the first letter, then the system would have to search all 50,000 login names to
find out which one was appropriate, probably would optimize that a bit, but I can tell
it it would be amazingly slow, it certainly was in the place I worked, so just be aware
if you, if you use it in that sort of environment, just be careful, now there are other forms
of the tilde expansion, and the first one is a tilde followed by a plus sign, what this
does is to expand the contents of the PWD variable, we saw that a little while ago, this
is the thing that tracks which directory you're in, and this is kept up to date by bash
automatically, so in my example, if you cd to the documents directory, assuming you're
in the top level, then echo space tilde plus, for my example, you get back, slash home slash
hpr demo slash documents, so that's just telling you which directory you're in, the main use of that
would just be to use it where you, you might use dollar PWD, the contents of the PWD variable,
there's an equivalent which is a tilde and a minus sign, and that's another variable which is
managed by bash, which is called old PWD when capital, that's used to hold the previous contents
of PWD, so you can, you can guess that when bash is updating this PWD variable, it saves its
current value into old PWD and then updates it, so the equivalent sort of thing where I show
you, changing directory to documents, then echoing tilde minus, and you see the previous directory,
assuming you started in the home directory, is slash home slash hpr demo, in my example,
then echo tilde plus, and then you get slash home slash hpr demo slash documents, so that's,
it's just really a shorthand for this old PWD, you could use it with the dollar on the front of it,
if you prefer, make it more explicit, I think it's probably why I didn't know about either of these,
because I've never used them, never really felt the need, but you never know, since I did learn about
them, I have used them once, I have to say, so the next tilde type of expansion, type of tilde expansion,
perhaps I should say, is where you can access the directory stack, now we looked at the directory
stack, in the show I did 1843, which looked at pushd and popd and does, which allow you to add, remove,
and display a directory stack, so a tilde followed by a plus or a minus, and then a number,
this refers to a directory on the stack, so remember you can look at the directory by using the
DIRS command, and if you follow that with a space minus V, we can see the stack with numbered entries,
so my example shows that, and I'm assuming that the directory I'm currently in, the one label zero
in the stack is document, and the directory that I was in before that was just playing tilde,
which is the home directory, and I'm assuming that in order to achieve this, I've used a pushd to get
to document, so in such a case, then the tilde sequence tilde one, or tilde plus one, both mean the same
thing, references the stack element numbered one, so if you type echo tilde one, in this example,
you will see slash home slash hpr demo, notice how it's expanded the tilde, when you look at it with
the DIRS command, the plain one, it shows it just as a tilde, but when you echo it, you see it
fully expanded, if you want to see fully expanded directories, you can put another option to the
DIRS command, DIRS space minus L, space plus one, would give you the same result, and the minus L
option asks for the full form, you might have been as confused about the next thing as I was when
looking at the DIRS command, you can also ask refer to stack elements in reverse order, and that's
what tilde minus number does, so tilde minus one, in the scenario we just looked at, would give you
the top of the stack, so it would start counting zero from the bottom of the stack, one for the
next one, there's only two in the stack, remember, so it would come back with slash home slash hpr demo
and that would be the same as if you typed DIRS space minus L, space minus one, so that is a fun
capability, I cannot see myself ever using it, and considering the conversations that followed on
from my show on these things, now there will anybody else, there you go, at least you know it's there,
now we've been looking at the tilde expansions in the context of file paths and directory paths,
so you would have thought that that would be the end of it, you would do ls tilde some directory,
and you'd see the contents of it, but you can also use it in the, when you're assigning variable,
assigning values to variables, so my example here shows a variable docs docs, and then that's
followed by an equals, remember that has to follow immediately after, I've mentioned this very
briefly, else with, and then the value it's getting, which also has to follow straight after,
is tilde slash documents, so in other words store whatever tilde slash documents expands into
the variable docs, then that's followed by an echo space dollar docs to look at the contents,
and surprise surprise in that variable we have slash home slash hpr demo slash document,
now bash has got a bunch of special variables, I wanted it to sort of manages all by itself,
and similar, there are similar things used by other packages and software systems, and these
sorts of variables sometimes contain the list of path separated by colon, there is a variable
called path pat in capitals, which is used when searching for commands, if you type some command
rhubarb, then it's going to search all of the, all of the directories in that path variable to find it,
and may not find it, of course, so in my example here I echo the contents of path variable,
and get a list of things like, use a local bin, use a bin, etc, etc, we might have a different
one when you come to look at your own, so bash allows the tilde expansion formats that we've
we've already seen to be included in such lists, so in my example I've taken the path variable
and added another thing at the end of it, and bash allows you to do that with plus equals form,
so I've got path, that's capital path, plus equals, then I follow that with a colon,
a tilde slash bin, then echo dollar path returns the same list as before, but with colon slash
home slash hbr demo slash bin on the end, so what that will do, the result of doing that,
and it could be useful thing to do, is if you have a directory called bin in your top level,
home directory I should say, then that would be the place to put executables that you yourself
have produced, so that means you can just type their command, their names, and as commands and
they would be invoked, now I make a point of pointing out to you that the value being assigned
to the end of path is not enclosed in quotes, usually you put quotes around the values that you
add because there are all sorts of pitfalls with the way that bash interprets these things,
and expands them, which is the topic we're in when we're talking about, so if they're quoted,
you don't get a lot of the expansion applied to it, so if it had been quoted, the tilde expansion
would not have happened and you would have stored the literal tilde slash bin, so now let's look at
the second topic that I have for today, and that's the subject of command substitution,
now in Linux and many other operating systems, you type commands and they return you some sort of
output, in the majority of cases, things that do this right to a channel that's known as standard
output, sometimes referred to as STD out, now it's possible to capture this output channel and to
use it in all sorts of ways, and that's really best described as command substitution, so if we take
for example the date command, this reports a date and optionally a time, and it does it in various
format, so to get today's date in ISO 8601 format, which I've commented is the only same format,
whichever one should adopt, so it starts from the largest to and shrinks down to the smallest,
so year month day, rather than month day year or month year day, or you know, it's like doing time
by minutes, seconds hours, anyway, sorry, I'm rounding, to do that, to produce such a date,
you would type date space plus, and the plus is the thing that says here's a format for the date
that I want date and time, maybe that I want printed, and in this case it's percent capital Y,
which is the year, a hyphen, percent lowercase M, which is the month, hyphen, percent lowercase D,
which is the day, now if you do that, you will get back a date in that format, I did this on the
fourth of November, so I got 2015 hyphen 11 hyphen 04, now this output could be captured in a variable
using command substitution, and my example shows a variable today, called today, in our case,
equals, and then dollar open bracket, and then that date that we just saw close bracket, now echo
dollar today comes back with the same thing, the same date, same format, so the thing dollar open
parenthesis, then a command close parenthesis, is a command substitution expression,
it's the recommended way to use it, there is an older form which instead of the dollar and the
parenthesis uses back quotes around the command, and I've given an example of how the same thing
could be done using the back quotes, but I'm not going to discuss them because it's a sort of old
form, I think it might well be phased out in due course, though there's a lot of legacy scripts
that still use it, so maybe not, but anyway I prefer to use the parenthesis, the dollar
parenthesis one, because it's a lot more flexible, so the text returned by the command is processed
to remove new lines, now the man page about this seems to say that embedded new lines aren't
removed, but in my experience it seemed that they were, so I'm not quite clear about that,
but if you know better let me know, so I adjusted the plain date command to put
output multi-line output for demonstration purposes, and in order to do that the sequence percent
lowercase n is a way of generating a new line in a date format, this time I've put quotes around
the whole thing because of the spaces in the format that I want to output, and my command is date
space plus open double quotes, today hypheness, today's date, space date that is is, and then a
percent n for a new line, and then the percent y, percent m, percent d thing we had before,
and what we get back is today's date is new line and then the date, remember we needed to quote
that because it's got embedded space in it, now if we use the same thing in a command substitution
and we get a different result, so in the example I'm using today again equals dollar open
parenthesis, and then that date command we just saw closed parenthesis, and then when I do echo
dollar today I get today's date is space to 015 blah blah blah, so the embedded new lines been
replaced by a space, so to finish this topic I wanted to demonstrate another use of this sort
of command substitution, so I'm assuming there's a file called words with one word per line,
and I wanted to construct a bash loop which processes this file, and all I'm going to do with it,
just for demonstration purposes, is to echo each word and its length, so I've got an example written
out in the notes, and it consists of a for loop for space as f o r space w, that's a variable in,
sorry, space in, space dollar open parenthesis cat space words closed parenthesis, then on the next
line do, then next line echo space open double quotes dollar w, space open parenthesis dollar open
brace hash w, closed brace closed parenthesis, closed double quotes, then the next line done,
so what that's doing is the cat is cat command is some is a way of it, it just lists the contents
of a file, and it's being used as a command substitution, so a for loop we'll see as it's executing
a list of words that came out of the file, there's only 10 in this case, we'll come onto that in a
minute, and it's simply setting the variable w to each word in sequence, the echo echoes the
contents of that variable, and then it uses the the way of computing the length of a variable,
as we discussed in show 1648, if you want to go back and look at that, there's a link with
little links in the notes to show its length, so I'll just put some typical output in the notes
words with a space and then a bracketed length of number character to the word, there is a fast
away, an alternative and faster way of doing this without using cat by the way, and I mentioned
this because it's in the man page, and I thought it was worth highlighting, instead of cat he
uses a less than sign, so dollar open parenthesis, less than space words, closed parenthesis is the
command substitution, which lists the contents, so this this was a real example, I like to always
build the the environment that I'm talking about, just to prove that I'm myself that I'm not
talking nonsense, so you might be interested to know how I generated the file of words, if you
not then you can just skip the end, the loop that I did I showed you the loop that I used,
and I did it this way because I wanted to just demonstrate some of the things we've looked at
over this this sort of mini series, so I've got a for loop which is for space i, space in,
space, and then we've got a brace expansion expression which is open brace 1.10, close brace, so you
should know what that means from the last episode in this series, it just generates the numbers 1
to 10, the next line is do to go with the 4, and then inside the the loop the first statement is
an assignment to a variable, this time it's w, w equals dollar open parenthesis,
shof s h u f, space minus n one, space, and then a file which is slash user, usr slash share,
slash dict d i c t slash words close parenthesis, that is pretty much a standard system dictionary
which contains a bunch of words is used by the very spell checkers and so forth that you can run
on Linux, so that the shof command is a thing that chooses a random line or lines from a file,
short for shuffle, so in this case I've told it to go to that file give me one line chosen at random,
so the next line consists of a modification to this variable w that's got the word in it,
and I'm doing this because the words in this system dictionary often have possessive forms,
so it'll have banana, and then bananas with an apostrophe s, I thought I'd just remove them,
and anyway it was a way of demonstrating stuff, so the expression I've got is w equals,
and then dollar and an open brace, w, percent, open square bracket, abaro, a hyphen z,
the lowercase, capital a hyphen z, closed square bracket, star, close break, so you should recognize
this, you should recognize this, you really should, as an example of the remove matching suffix pattern
as I discussed in show 1648, okay maybe you don't remember it, but it all it does is to look for a suffix
inside the variable and remove it, where the suffix is defined by this percent and then some stuff,
what I've done here is to say the stuff that it's looking for to define the suffix is any
character which is not in the range lowcase a to z, another case a to z, so in other words something
that's that's probably going to be a hyphen, going to be an apostrophe I should say, and then the
asterisk after that means anything else up to the end of the word, and so I asked it to remove that,
so the banana apostrophe s would be trimmed down to banana by that mean, thought you might like that,
or not, anyway I liked it, I'm having fun, so the last line of the loop is the the done, which is
the way you close a loop, and that's followed unusually by a greater than sine, space greater than
space, then words, words is the name of the file, and the, oh sorry I forgot one, on one line,
there was an echo I forgot, which is echo dollar w, which echoes the the the the the
word that's been pulled out of the file and then trimmed the the tail end trimmed off if necessary,
so the entire loop sends its output, which is a list of 10 words, because it rates 10 times,
into a file called words, and you will know that because the earlier loop we wrote read that
file, this uses what's known as redirection, and I've penciled in some notes about doing show on
the on the subject to redirection, later on in this sub series or whatever you call it,
having written this I thought oh maybe there's way I can make this better, because the shuff
command can return multiple random words at a time, so I was able to change the loop to my final
example, which is four space, w space, in, and then the command substitution is is dollar shuff,
SHUF space minus n10 space, and then the name of that system directory dictionary that we saw
before closed parenthesis, so what that is saying is loop as many times as you get back words from
shuff, and shuff has been told to return 10 words, and put each word into the doubly variable,
then the loop just contains the one line, which is an echo, an echo contains that remove matching
suffix pattern expression that I rambled on about a little while ago, so that's simplified the
whole thing quite a bit, and the whole loop is redirected to the file word, so if you run that,
you just get a bunch of random words in that file, and it overrides the previous content by the way,
but I'm preempting my later chat about redirection if I'm going to too much detail there,
all right so that's about it, that took a lot longer than I thought it was going to, I thought
these are two lightweight subjects, but according to my recorder I'm now 36 minutes in, hopefully I
can trim that down a little bit by some of the moving some of the gaps, anyway enough, you'll find in
the full notes there are some manual page extracts, where I've given you the gist of the
what you find in the bash manual, and there's a bunch of links that point to previous shows and
other bash references, I hope you found that useful, and speak to you later, okay bye!
you've been listening to heckaPublicRadio at heckaPublicRadio.org
we are a community podcast network that releases shows every weekday, Monday through Friday,
today's show, like all our shows, was contributed by an hbr listener like yourself,
if you ever thought of recording a podcast, then click on our contributing to find out how easy
it really is, heckaPublicRadio was founded by the digital dog pound and the infonomicum computer club
and is part of the binary revolution at binrev.com, if you have comments on today's show,
please email the host directly, leave a comment on the website or record a follow-up episode yourself,
unless otherwise status, today's show is released on the creative comments, attribution,
share a like, 3.0 license.