Episode: 1197
Title: HPR1197: What I do with bash scripts
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr1197/hpr1197.mp3
Transcribed: 2025-10-17 21:26:34

---

Hi, this is John Culp in Lafayette, Louisiana, and this will be my very first solo podcast
for HPR.
I did one with NY Bill at one point about shopping at Goodwill, and I'm answering now the
call that Hacker Public Radio has recently put out for new podcasters because apparently
there are not very many in the queue.
And the trouble with me is not necessarily the desire to record a podcast, but the lack
of topic.
And so I decided I would go ahead and talk about something that interests me, and hopefully
it will interest listeners as well.
I'm not an IT professional.
I am a music professor in Lafayette, Louisiana, however I do have pretty good skills on the
computer and in Linux and so forth.
And what I wanted to talk about today was the way I use scripting in my everyday life.
I wrote my first script, I don't know, four years ago maybe, when I was listening to
a podcast by Chess Griffin called Linux Reality.
And he had a couple of episodes about scripting and starting out with just very basic things
where you stored one or two commands in a file and then ran it and let it, I think my
first script was something like it listed the contents of my documents directory and
stored that output in a file.
And very soon I got a little bit more sophisticated where I could grab command line arguments and
do things to them and stuff like that.
And now it's at a point where I really don't know what I would do if I could not use
the scripts that I've written and that make my life easier on a day-to-day basis.
The scripts I use fall into a few different categories and I've tried to kind of list
a few in each category.
I would not dream of talking about all of the scripts that I use because there are more
than a hundred in my personal bin directory.
But one category is the syncing slash backup type of script.
Another one handles compiling.
Another category is conversions of files from one format to another.
I have some system type scripts that check on the status of various things and then I've
also got fun scripts where the sole purpose is to do something fun just because you can
with a script.
I'll get to those last.
The first category, syncing and backup, I use these all the time pretty much every day.
I've got a couple that will make backups of important files like my password database
for keypass X.
Whenever I add a new password to entry to that, I will run my password backup script and
it will take the key file and copy it to a remote location where I use it like on my
work computer or where I keep a backup of it.
I also have one that I use to copy the public key directory that I have or I guess the
public key ring that I have for my email.
A bunch of us who met each other on Identica and status net have formed an encrypted email
mailing group and whenever somebody new joins the group and my key ring changes, I will make
a backup of the public key ring and so I have a script that does that.
It uses an rsync command and actually it's a secure copy, not rsync.
But I just in the script, I tell it where the source file is, what the destination is,
and then I give it the command.
You could type this out as a command but it's hard to remember the IP address and which
port the remote server uses and all that, it's just much easier to store it in a script.
Same thing with my address book, if I change something in the address book, I'll run my
address book sync script and that will copy the source file to the various places where
I use the address book.
One of the most important ones to be on a daily basis is my iPod syncing script.
I have an old iPod, I'm very proud of the fact that my iPod is from around 2005 and
I'm still using it after two battery changes and for the last four years or so running
open source firmware on it, the rockbox firmware.
But every day I get podcasts and then I sync it up.
Now this one is a little bit more sophisticated than the other syncing scripts because I run
a couple of sanity checks.
The first one checks just to make sure that the iPod is actually mounted.
So it'll, it goes in and looks in the media directory or somewhere.
I can't remember exactly how.
I've gone through a couple of different ways of finding whether the iPod is mounted or
not.
Anyway, it checks.
If it doesn't find it, it just exits with the message, hey, you need to plug in your
iPod.
It's not there.
If it does find the iPod, then it runs an R sync command that will sync up my podcast
directory with the iPod and that includes not only copying stuff over but deleting podcasts
that I've already listened to.
So that one is useful.
Another one that I came up with that I really like, I call stick.
I actually wrote a blog post about this one.
The stick script is one that I use to stick files on servers.
For me, this was a way to make the secure copy command easier because secure copy is an
excellent way to transfer files from my local machine to the various servers that I have
running or to my office machine or whatever.
But secure copy is difficult to type out every time because you have to remember either
the IP address or the host name or whatever.
And I also have SSH running on different ports on almost every machine and I just can't
remember what port they are all on.
And so what I did was I wrote a script that will take the first command line argument.
So the command will be stick and then I will say stick to something so it will be stick
space server name and whatever the server name is will be in the file somewhere and so
I have an if then, what do you call it, an if then statement I suppose.
So if the server name is x then port is y and then it sets all of the variables accordingly.
And then it will transfer that file to the home directory on the remote server.
This has been enormously helpful to me because if I need to stick a file somewhere, I just
go stick file name and then server actually it will be the second command line argument.
The first command line argument is the file name that I'm going to be transferring.
So it will be stick fubar.text server and that's how it works.
I really like the stick script.
I use it all the time.
Now the another category of scripts that I have is for compiling stuff.
I do a lot of documents and my music notation software all use source files that then need
to be run through some kind of converter or something.
So I use lilypon for my music notation files and it's possible to run a lilypon command
at the command line by typing it straight out without too much trouble but there are lots
of command line options that you can put with lilypon and I like to use a number of those
and it would really be a pain to have to remember what they are and to type them out every
time.
I have a script that helps me with those and it would be pretty boring to go into all
the details of that but it is save me tons and tons of time having that.
There's also another lilypon related one that I use for running the lilypon book command
which also has many command line options and you have to specify output directories and
output formats and all of this and I've scripted all of that where I just have to type a single
command and then the first command line argument is the file upon which I'm running it and
that really simplifies things a lot.
I have similar scripts to deal with my markdown files and my lot tech files and stuff like
that.
Another category of scripts I have is for text manipulation and these are ones where
I'm trying to save either me or someone else a lot of time by automating something that
can be automated.
For example, just a few days ago I was faced with the task of reading nearly 100 essays
written by my students in a big music appreciation class and when I go to look at those essays
on Moodle what I have to do is go to where all the essays are and click on a link which
then has a pop-up window with the students essay in it and then right now our Moodle theme
is all messed up so I also had to maximize that window to be able to see all of the text
and I have to read it, click out of the window, enter a grade, click on the next essay and
this ends up being quite a lot of clicking and maximizing and closing and I did not want
to have to do that for 100 files but there's also an option there to download all of the
essays in a zip file so I tried that and found that it did not download all of the essays
as one big file but as 91 different HTML files.
Now this is not a whole lot better because I would have to open up every single one and
close it and read it and so forth.
What I did was I wrote a script that will cycle through every single one, rename the file
so that it removes the spaces in the file name because Linux and Unix type things don't
like spaces and file names so the first thing I do is get rid of all the spaces and file
names then I add some heading material for the HTML like I tell it that I want to use
the UTF-8 character set and that I want the maximum width to be 40 m dash m space I'm
not sure what that is I say max width is 40 em web developers will know what that is
because I just found it online somewhere but it limits the width of the text and it
makes it more readable and so I put that at the very top and then I just start cycling
through every single one of these HTML files and appending the contents of it to the file
and at the beginning and end of each of the students essays I also put the students name
saying begin you know John Doe then their essay and end John Doe and then put a horizontal
rule so I don't get one essay mixed up with another so anyway and the upshot is after
about less than a second it's finished concatenating all of these essays in one big page and it
opens it up in my browser so I can just scroll straight down the page and see all of them
and this really made my life easier see I have another category of scripts that are for
converting things from one format to another and the the tool that I found amazingly useful
for this kind of thing especially when dealing with images is the suite of tools called net
pbm just perfect for scripting there are tools in there to convert just about any kind of format
into any kind of other format and so I have one that I call image to image where it'll take an
input file that is some kind of image png or jpeg or jiff or whatever and then you can choose
whatever output format you want and it will run the conversion for you I've got one that I call
thumb that I wrote when I wanted to have an easy way to make a thumbnail image to upload as an
avatar for user forums or things like that and it also uses the net pbm suite of tools and it
asks for user input like it'll ask what the input format is and then ask how many pixels wide
you want it to be and then it'll just run it for you there's one that I wrote for rotating an
image it asks you which direction you want it to rotate 90 degrees positive or 90 degrees negative
and it will do that for you another conversion script that I just did recently that I'm very pleased
is one that will take a markdown input file and convert it to lottex now this is not entirely
my own work there's a guy named fletcher pinney wrote the program called multi markdown which
uses markdown syntax but extends its flexibility a bit by making other things possible like alternate
output formats besides just html one of those output formats is lottex but what I found is that
when you run that and use lottex output it does not produce an output that is ready to compile
to sorry to compile with pdf lottex or anything like that it is missing the preamble it's missing the
end document command at the very end and so forth so I wrote a script that will put in the preamble
that I want then it will run multi markdown and redirect all of that output and append it to
the file that has my preamble and then it will put the closing end document and so forth but then
before it is finished it will run back through and check to see if I've used any utf8 international
characters like accent to characters or characters that have umlauts on them or things like that
because lottex does not deal with those very well you have to format them in a specific way using
curly braces and accent marks and things like that and so my script will go through and check to see
if I have any of those characters and if so it will convert them to be ready for lottex to compile
so that's one I'm really proud of and let's see I also have some scripts for doing system
stuff one of my the ones I use the most is called c just se and that's when I want to see what the
process is of or the process id is of a certain process it's basically it takes ps
space a-u-x pipe grip and it will store all of that part of the command and all I have to do is say
let's say I want to I don't know thunderbird or firefox or something is frozen up and I need to find
the process id to kill it I'll say see thunderbird and it will run that and give me the output
I also wrote a script that I call my ip that will show me what my ip address is on the various
network interfaces that I have at the moment it essentially uses the ip space a-d-d-r command
and strips out everything in there that is not the ip address
is one thing that's always kind of annoyed me about the running the command ip space a-d-d-r
is that it does give you the ip address but there's all kinds of other information that I don't
necessarily want and so my script will just tell me the ip address and which interface it's on
let's see also wrote a script for updating my moodle instance I have my own instance of
moodle at home and it's running from git it's tracking the stable branch and I found that it was
difficult to keep it updated because not only is the main tree tracking git but I've got a few
modules that are also tracking git in different places and so I'd have I first of all could not
remember which modules they were because many modules are in the core and then others are ones
that I've added and I couldn't remember which ones were which so I just stored everything in a script
and then I run the script and it'll go into each part of the file tree that has a git
um a git repository and then check and see if there's any new code if there is it'll grab it
then when it's gotten all the new code it'll ask if I want to run the update script for moodle
that that upgrades the database tables and so that one has been a big help
another script I use every day it's not a system one at all but has to do with media is mash
potter mash potter is chess griffin's modification of bash potter and I like mash potter because
of the different file naming conventions but I've also added some modifications to it
and mostly but because of a single podcast that I listen to every day it's a sports radio
podcast and for whatever reason in the last two months or so when the sports broadcasting guy
changed networks and they started using a new file name and convention that is completely random
as far as I can tell and so it'll pull in the episodes but the file names give the media player
no indication as to which episode it's supposed to come first well I mean this podcast actually
has three different files for it every single day it's a three hour long show and sometimes it'll
get the file names out of order and this really annoyed me so I added some lines to mash potter
that would go into that specific directory where the sports podcast went and use the MP3 info
command to get the ID tags from it and use that information from specifically from the title tag
to rename each of those files in a sane way and so now once my script is run
I have the three episodes every day that are renamed so that they appear in the correct order
in my iPod I also wrote a script or another few lines in there
that checks for shorter portions of that same podcast so what they do is every day they'll give you
the three hours of the show but also they'll put in extra smaller segments of the show just with
interviews of different people that the guys talk to that day and I don't want the shorter ones I
just want to listen to the whole show straight few sorry straight through and so I wrote a couple
of lines and added it to mash potter that will go in there and check the file size of each of those
things using the find command and when it finds any files that are smaller than 20 megabytes it
just deletes those and leaves the ones that I want that's been a very helpful script
now the last category of scripts that I use are just the fun and silly ones I have a weather script
that will well it checks the weather at I forget where it is um let me see here
use acuweather.com I use the e-links text-based web browser and just dump the contents to a temporary
file and then I use grip, awk, said, whatever commands like that to extract the information I want
and then format it in a way that will fit in like a micro blog kind of post and then I have the
information posted to my status net timeline by my cat my cat dingle he's got his own account on
there and when I type the weather command it does all this stuff finds the information and then
posts a message from dingle on my timeline telling me the weather conditions and the forecast so
gives me some information but it's also just meant to be kind of a fun script and one that you do
just because you can I have another one related to weather where I use cow say cow say is one of those
great unix programs where you type in a string and a cow in asky text will tell you what you typed
in well I found out that there are more than 40 different cow say asky images and so I wrote a
script that will randomly choose a number first what it does is it locates all of those cow say
image asky files and then counts how many there are and then chooses a random number that will fall
in that range and uses that one to choose the image to use for the day and I've got this running on
a script that every morning I think at 5 a.m my server runs this and then emails me a weather update
just telling me what the temperature is and you know it's it's just kind of fun I thought it was
fun to be able to script an email message and to learn how to do a random number and choose a
different cow let's see I've got another one that my cat does is generates a random password and
and post it to my status netline I call this security kiddies password of the day and so I what I
can do with that one is by default it will choose a 24 character password but I could supply a
command line argument of any number I want so I could do a password space 16 and it would generate
a 16 character random password and then post the message to my timeline from my cat
what else here I've got another script related to cow say which will just cycle through all of the
various cow say files and have the cows and images say the same thing for everyone just so you can
see what all the images are that one's kind of fun I wrote a couple of scripts that were for either
me or my kids to practice memorizing information for example when my kids were younger and they
didn't know our phone numbers I wrote this script that will ask them whether they want to try their
moms or their dad's phone number and then they would type in the number and it would tell them
whether they got it right or not if they missed it it would ask them to try again or if they wanted
to exit and so this is how they learned how to remember our phone numbers and then I have one
that I call the password test whereas if I if I've decided that I want to start using a different
password that I have to type in I mean most of my passwords are I keep in my password database and
I don't even know what they are but if I need to learn a new password that I'm going to have to
type before I start using it I will test myself on it dozens of times to make sure I'm not going
to forget it and so I have a script that asks me to type it in and then tells me whether I got
it right or not and I think that's really about it I will probably post some of these scripts on
my website for you to view I don't think it would really be necessary to post all of them
unless somebody just really wants to see them but none of these really go that deep into script
foo I'm not that good at it but I just find that I can I've learned how to do certain things and
they really help me in my day-to-day life even though I'm not an IT professional so I hope you
enjoy that and maybe I'll be back some time to do another podcast if I can think of what to talk
about see you later
you have been listening to Hacker Public Radio at Hacker Public Radio does our
we are a community podcast network that releases shows every weekday Monday through Friday
today's show like all our shows was contributed by a HBR listener by yourself if you ever
consider recording a podcast then visit our website to find out how easy it really is
Hacker Public Radio was founded by the digital dot-pound and new
phenomenon computer cloud HBR is funded by the binary revolution at binwreff.com
all binwreff projects are crowd-responsive by lunar pages from shared hosting to custom
private clouds go to lunar pages.com for all your hosting needs unless otherwise stasis
today's show is released on the creative commons attribution share a line