hpr-knowledge-base/hpr_transcripts/hpr1938.txt

Episode: 1938
Title: HPR1938: How I prepare HPR shows
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr1938/hpr1938.mp3
Transcribed: 2025-10-18 11:29:30

---

This is HBR episode 1938 entitled How I Prepare HBR Shows.
It is hosted by Dave Morris and is about 26 minutes long.
The summary is, I own my own tools for preparing my HBR Shows.
I talk about them in this episode.
This episode of HBR is brought to you by AnanasThost.com.
Get 15% discount on all shared hosting with the offer code HBR15.
That's HBR15.
Better web hosting that's honest and fair at AnanasThost.com.
Hello everyone, this is Dave Morris.
Today's show is entitled How I Prepare HBR Shows, which is a bit meta, but there you go.
So why am I talking about this?
Well, I've been contributing shows to HBR since 2012, according to my records.
And in those far off days, we sent everything via FTP, and we had to name the files with
a combination of our host ID and numeric thing, our name, or handle, whatever we're using.
The slot number, HPR1234, the title, and it was all rather complex with hyphons and
underscores, meaning special things and so on.
The show notes had to contain a chunk of metadata in a predefined format, and that was to signal
various things like weather, it was explicit or not, and all these types of things.
It was quite complicated, I certainly found myself making a number of mistakes with the
naming and metadata stuff.
So whenever I have a problem that is a little bit too much for my old brain, I write a bit
of software, or indeed make a device to help me with it.
So I started developing a bash script in mid 2013, which I called somewhat unoriginally,
HPR underscored talk.
I used bash to do this, since I thought it would make something which had few dependencies,
fairly small footprint, maybe I could share it, and it may be useful to others.
The script grew and grew and became increasingly complex, and I found I needed to add other
scripts to the toolkit as it became, and to resort to pearl and various pearl modules
to do all the things I needed to do.
In 2014, Ken changed the upload procedure to what it is now, which is much better.
It's a better design, does the way with the need to name files, and put metadata in them,
and all that good stuff.
But it left my toolkit a little bit high and dry, so I shelved plans to release it at that point.
But carried on using it, I've been messing around with the hacking around,
changing the way it works to a fair degree, debugging it, and so on and so forth.
The outcome is it's a little bit idiosyncratic, quite complex.
Not really a thing you'd want to give to somebody else, quite personalized to my needs, I suppose.
But I thought I'd put it up on GitLab, which I have, there's a link to it in the show notes,
and I thought I would describe what I'd do with it, just in case anybody was interested in it,
or wanted to just have a look at it and see what I, the way I do the various things I'd do.
So it's given an overview of the script.
It's called HBR underscore talk, and it has to be put into a directory where the
shows are going to be stored. The principle is that each show and its related files are stored
in their own sub directory. The script derives the name of this directory from the title of the show
that it holds. That's really a hangover from the previous days, but I thought it was a reasonable
way of going. So I keep my scripts, I keep these scripts that I'm talking about, and the HBR
episodes in a directory under my home directory, there's HBR is the top level directory, and then
underneath there is a directory called Talks. I've got quite a lot of HBR stuff on my main workstation,
as you might gather, being a bit of an administrator and a messer about your own Wrangler.
Anyway, there's no problem where you put them, as long as all the scripts are in the same place,
and the shows are all created in directories underneath, and you own the account you're using it
from, can read and write to this directory tree. So the HBR Talks script takes one or two arguments.
The first is an action word, think something like install or create. You can abbreviate these
down to the shortest unique form. That's the thing that sort of is an imperative to the script.
The second argument is there in certain circumstances, not in others. It varies mostly with whatever
it is you're doing. If it's present, it can be the name of the directory holding your show,
or it can be the title of the show in quotes, where you can just leave it out and the script will prompt you.
There is also a minus H option, which will display help. So I put, I describe the workflow
that I go through when I'm preparing a show or episode. I tend to call them shows,
or people call them episodes, so I'm sort of using the two terms interchangeably.
So the first thing I would do would be create a new new episode. I do this sometimes just when I have
an idea for a show, something I'm going to do in the future sometime, and I just create the sort of
template, the framework, as a placeholder, and then I can go and fiddle around with it later on.
During the creation step, the script collects all the metadata relating to the show,
which again is a hangover from the old system, but I found that it's useful. It asks questions
and sets defaults where the answers aren't clear yet. Things like, what slot do you want? Well,
you don't know yet, you haven't reserved it unless you're a lightning fast with producing episodes.
So having created a show, then you come back and mess around with the title. Don't like
where I phrase that, change it, you can change the summary, you can add and subtract tags and so forth.
You can do that as many times as you want all the way through the lifecycle of that episode.
So the shows I create consist of brief notes for the main page,
and I usually go for some sort of longer note format, which is available linked off that page.
The one you're listening to now is configured in that sort of way. My notes are always prepared in
markdown, and then I use HPR talk script to convert them to HTML before uploading. So I might have
some supplementary files such as examples of scripts or whatever, some pictures, that type of thing.
And if I do have that, then then those files and the long notes are all packaged up into a
compressed tar file for upload. So I only ever send three files to the server. That's the main
notes, the tar file, if appropriate, and the audio. Since I'm generating HTML from markdown,
I've added features to the HPR talk script aid me running this process. First of all, I create
boilerplate markdown files with the script, so I don't have to worry about the formatting.
And so forth when I start them up, and I generate a make file, which allows me to automate things
using GNU make. Since I use VIM to edit my notes, I generate the configuration file so that I
can run the session plugin within VIM. Session is a thing that lets you open multiple buffers or
hand or windows, and then save that configuration for the next time. So I use that for everything,
and I've got a thing that builds a session configuration. So all I need to do is to go, I use
GVIM mostly. Open GVIM, click on the session menu, the top of the screen, and top of the window,
and it will give me a list of all the sessions I have, and one will be the show that I want to
edit. So I just do that, and everything is loaded up as it was previously. When I saved it,
or indeed if it's the first time, then everything's there in some default form. If I'm
including multiple files in my shows, such as pictures or scripts, I list them in a manifest file,
and the HPR talk script allows me to create this file and populate it, and the make file uses it
when generating the tar file. So as I work on the notes, I rebuild the HTML version and view it
in a browser, and the build process can be run out of VIM using the make command, which is available
within VIM on its command line. I can also go to another window and run the HPR talk script,
telling it to build some aspect of the notes. Now, so you can view this, if you've got references
to other things in your notes, like pictures or whatever, then these are going to be references
like file references, not full blown URLs. And you want to be able to do that, because you want
to see how it looks. But when the notes go up to the HPR server, they're going to be referring to
files on that server. So I build in a facility whereby if you type the command make space final,
F-I-N-I-L, or you run the appropriate option within the HPR talk script, it will build everything
to work on the HPR server. Now, this toolkit doesn't do anything with the audio. This is generated
independently. I use perhaps audacity or a portable recorder. I'm using my Zoom H2 just now for
this one. I save the raw audio in the show directory and edit it with audacity, and then finally
export the flack file to the show directory. And I name it according to the conventions that
the script uses. I use audacity to save generic audio tags. It's possible to configure audacity to
do this for you during the export. But I do have a tool for generating specific tags automatically
from the show configuration file. I haven't included this as part of the GitLab configuration,
because it's quite complex and there's a whole raft of dependencies there. If anybody's
interested, I could include that because all of the components are available, but it takes a little
bit of building, that's the main downside. So as the components of this show accumulate, the files are
registered through the script. And like I said before, the primary elements that are known to
the script in the configuration file are the main notes, the supplementary files and the long notes
bundled into a tar file, and that's defined by the manifest file I mentioned before, and then
there's the audio. So three main components. So once everything is ready to be uploaded, I reserve
a slot on the calendar page, as usual, and then use HPR talk to change the configuration file,
change the slot entry in the configuration file. This causes all of the files to be renamed
automatically, and so the convention is that they should be called HPR, 1, 2, 3, 4 or whatever
on the front of them. So that's done by the script. And it also causes the HTML references in the
notes to be changed. So when I do the final build, I use Ken's upload form and fill it in manually
with the details from the configuration and with the main notes, which I simply cut and paste in.
Then I use HPR talk itself to perform the upload. It uses curl underneath to write to the FTP server.
So it's using FTP for the upload of everything other than the main notes. Actually in the
current version, it sends the main notes again, but that's something I'm going to fix, and you
can would prefer that I didn't do FTP uploads, but it's just I just found it so much more convenient
to do things through this script. So I thought I would just skim through the main features of the
script, most of which I've covered in some detail, as I talked about my workflow. So first thing
that you would do if you if you were using the script would be to install it. Basically you just
put it somewhere, but then you need to run its install function because it has a bunch of
configuration files. Some of these are populated by itself and some of them need input from you,
so it needs to know your host ID and your host name and so on and so forth. And it also needs to know
the FTP upload details, the FTP password. If any of these things change and the only one likely
to change is the FTP password, then you can simply run the install process again and it will
allow you to make modifications. So you can create a new show with this through the create
function that you'll be prompted for a whole bunch of parameters, as I mentioned, and the show
directory will be created. At that point the show number or the slot is unknown and so the files
that are created in the show directory are called HBR followed by four underscores and then these
get renamed when a slot has been reserved and you tell the script. There's quite a lot of things
you can do to change the configuration of the show, including change the title or the. If you change
the title is going to change the name of the directory too and you can update the summary or you can
that's where you put the slot in when you've decided what it is. There are a whole bunch of more
advanced features that I've added which let you create the main show note template or the
and the full show note template. It lets you create and populate the manifest file and it creates
and make the make file to drive the whole thing. During the make file creation you're asked
things about how you want to run things. One of the options is for example would you like to create
ePub notes from from the thing at the end and this business of having a VIM session I mentioned
before can be you can create the session configuration at this stage if you want to and if it's
appropriate to you. Building the notes is another function and it runs various forms of make so
you can build just the main notes. That might be all you have of course. You can build everything
which includes the main notes, the supplementary notes or the full notes, ePub notes and if there's
a tar file which there will be if you've got full notes at very least it will build that. It'll do
it in a way that you can view your HTML locally. Then there's build final which is same as building
everything but it it fiddles with the URLs and there's also a feature to refresh all the files
because make needs the files to have changed and you don't always need to change them. You don't
always intend to change them before you you build them. You need to register all the various files
that you have with the show in the configuration file in the script and let you do that and when
you've done all of that a lot as I've already mentioned you you can I used to term release. It should
have used upload it or I chose release. You can you can upload it release it via FTP. It checks
if registered everything and the slot is allocated and so on and so forth. There are also some
reporting functions here which let you look at the status of a show. So if you've got a whole bunch
of them that you've started working on but haven't done anything with for a while and I certainly
have plenty of those then you can run the status function on a given show and it'll tell you
some stuff about where you are. There's also a summary function which scans through all of the
shows you've prepared. Fishing out only those that you haven't uploaded and it reports on their
current state. So if you have several of them on the go it's useful for that. Again this is
done for me because I've got something like I don't know a dozen or so shows in an intermediate
state. I thought it mentioned a brief bit about the show notes that this thing expects. They're
in Markdown format but we're using Pandoc to process the markdown so it's capable of using the
extended features that the Pandoc allows and I've made certain assumptions about which of these
you would use. This is something that perhaps could do with some further work and for example
it makes it makes certain decisions about the formatting of the notes. The main notes are always
built as an HTML fragment because they're a piece of HTML that's to be fitted into another pre-existing
template that's displayed on the HBR website. The extended notes if you're using them are
built standalone so it's complete webpage all of its own and the extended notes always refer to
the HBR site for the CSS so they look similarly to the same fonts and colors and
size of page and so forth. There's also the EPUB option where I've made certain assumptions about
the layout. Still really working on this because sometimes the EPUB format doesn't go very well with
the the sort of notes that I've produced for example scripts and that type of thing.
One of the aspects of the notes is that they're really a template. They pass through a pre-processor
before being handed to Pandoc and this is a pulse script which interprets expressions
in template toolkits syntax and this pre-processor is given arguments which are the names of the
various files. So the main and the supplementary notes are both passed through a pre-processor before
being handed to Pandoc. The pre-processor is a pulse script which interprets expressions
in the template toolkits syntax. I've put a link to template toolkits in the notes. This allows
strange sequences of brackets and percents in the notes to be replaced by arguments to the
to the script. The arguments that are given to the pre-processor are things like names of the
various files like the extended notes file and the contents of the manifest file. That's
appropriate and this simplifies the process of linking to supplementary files and images and
allows the generated URLs to change depending on whether the HTML is for local viewing which case
it's a local file name or whether it's going to be the final version which case it's a full HTTP
URL referencing the HPR website. So I've put an example of what I tend to do and will
have done in these notes. I haven't actually written it yet but it will contain a line in the main
notes which say something like I've written out a moderately long set of notes about this subject
and these are available here and then the example shows an open square bracket and then another one
and a percent sign space ARGS dot zero space percent closed square bracket closed square bracket
open parenthesis then another sequence of open square bracket percent args dot zero percent
closed square bracket and a closed parenthesis. What that is the the thing open square bracket
percent whatever percent closed square bracket that's an expression which substitutes a variable.
The variable is called args ARGS meaning arguments is actually an array and I've requested the first
of first element of this array and this is the first argument that's been given to the pre-processer.
Now that expression is repeated in the example and the two copies of it are inside the markup
markdown markup for a link so you put text in the first square brackets and then you put the URL
in the round bracket. So that's that's very useful. It allows the the script and the the various
elements of my toolkit to handle this for me. There's another example I've recently prepared a show
with multiple images and I inserted them using a slightly different method. You'll see there's
an example which starts with an open square bracket percent minus or hyphen space default
in capital space i equals zero then a hyphen percent closed square bracket. The minuses the hyphons
inside of the percent tell the pre-processer not to generate new lines when processing this particular
line so it's just a way of stopping it from adding in lots of unnecessary new lines which which
might be significant in certain places. So the example shows the way in which you would generate an
image in markdown so it's an exclamation mark and then in square brackets the text preparing
carrots one closed square brackets then in parentheses one of these args things. So in this particular
case it's open square bracket percent args dot dollar i. So what that means is use whatever the
value of i the variable i in the template is so it would be zero in that particular instant and
then the rest of it that the text won't read at all. Then the second example of an image shows
inside the template toolkit expression open square bracket percent then we've got i equals
i plus one. So if incremented the variable i semicolon is a pearl light syntax and then we use
args dot dollar i and then we close the the thing with the percent in the closed square bracket
and what that does it it plants args one the second element fed to the the pre-processer. So
that makes it a little easier to shuffle things around you don't have to worry about which which
image is which. The only thing is that the images the files or whatever it is you're working with
here are numbered according to the order that they were fed to the pre-process and the order is
defined by the the file the manifest file. This made it a lot easier to handle when there were
3435 images to place in note. So in conclusion then this toolkit does a large amount of what I want
when I prepare hpr show and it saves me from making the sort of mistakes that I tend to make.
The present design still shows lots of marks of its origin and the ways the way things were done
back back in 2013 and so forth and well before the current mechanism the submission mechanism.
I'll probably fix that in time I definitely will fix that in time just for my own benefit.
I'd like to automate the completion of the submission form but I'm not sure I know it can be done
but whether it's a good thing to have script effectively scraping the contents of that form
and writing stuff into it and so forth I don't know. I'm not quite sure what the security
implications are just at the moment and I'm not sure that Ken would be overly yet thrilled at
doing it that one. So that's something I'm working on anyway thinking about. You're welcome to check
out the scripts if you want to and see whether it's anything of any interest to you. You can even
just go and browse the stuff and that's the point at which you decide you don't want
anything to do with it. It's on GitLab and there's a link to it in the note. There's not a great deal
of documentation on the GitLab site at the moment but I'm gradually adding to it as I go along.
If I hear of anybody who's actually interested in messing around with it then that will give me
the incentive to improve on the documentation. If you do have a go at using it don't hesitate
to contact me if you have any problems or questions and if you have suggestions about how to
to move forward with this then that would also be appreciated. So I hope you find this useful. Okay, bye.
You've been listening to Hecker Public Radio at Hecker Public Radio.
We are a community podcast network that releases shows every weekday Monday through Friday.
Today's show, like all our shows, was contributed by an HBR listener like yourself.
If you ever thought of recording a podcast then click on our contributing to find out how easy it
really is. Hecker Public Radio was founded by the Digital Dove Pound and the Infonomicon
Computer Club and is part of the binary revolution at binrev.com. If you have comments on today's show
please email the host directly leave a comment on the website or record a follow-up episode yourself
on this otherwise stated. Today's show is released under Creative Commons
Attribution Shared Light 3.0 license.