Merge pull request 'I112_Add_support_for_a_different_php_basename' (#113) from I112_Add_support_for_a_different_php_basename into main

Reviewed-on: #113

GETTING_STARTED.md

@@ -0,0 +1,202 @@
+# Clone the repository
+
+If git is not installed on the operating system, please install it now 
+(see the git documentation for [instructions on installing git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)).
+
+To retrieve the code from the repository on anhonesthost.net (a username and 
+password are required), run:
+
+`git clone https://repo.anhonesthost.net/rho_n/hpr_generator.git`
+
+To retrieve the code from the repository on gitlab.com, run: 
+
+`git clone https://gitlab.com/roan.horning/hpr_generator.git`
+
+On success, an "hpr_generator" directory will be created in the folder from 
+which the clone command was executed containing a local copy of the git repository.
+
+# Install required Perl modules
+
+Installing the Perl modules is the most finicky part of the installation process. 
+The needed Perl modules can be found using the operating system's package 
+manager or using the modules found in the 
+[Comprehensive Perl Archive Network (CPAN)](https://www.cpan.org/).
+
+## Installing modules on Debian based Linux distributions
+
+Run command (tested on Debian 11):
+
+```
+apt install libconfig-std-perl \
+	libtemplate-perl \
+	libtemplate-plugin-dbi-perl \
+	libdbd-sqlite3-perl libdate-calc-perl \
+	libtie-dbi-perl
+```
+
+## Using CPAN to install the modules
+
+A cross platform method to install the needed modules is the Perl CPAN application. 
+Make sure both the [make](https://www.gnu.org/software/make/manual/make.html) 
+command and the [cpan](https://perldoc.perl.org/CPAN) command are available. 
+Install them using the operating system's package manager, or from source.
+
+Run commands:
+
+```
+cpan Config::Std
+cpan Template
+cpan Template::Plugin::DBI
+cpan DBD::SQLite
+cpan Date::Calc
+cpan Tie::DBI
+```
+
+# Create the HPR database
+
+The hpr_generator relies on information from a database to generate many of the 
+files for the website (for example: index.html, series/index.html, 
+hpr_mp3.rss, etc). This data is available from a MySQL dump file found on 
+hackerpublicradio.org at URL "https://www.hackerpublicradio.org/hpr.sql".
+
+The first step is to download the hpr.sql file. This can be done using your 
+browser, or by running one of the following commands:
+
+`curl https://www.hackerpublicradio.org/hpr.sql --output ./hpr.sql`
+
+or
+
+`wget --directory-prefix=./ https://www.hackerpublicradio.org/hpr.sql`
+
+## Creating an SQLite database file
+
+The SQL of the hpr.sql file must be converted from MySQL specific statements to 
+SQLite specific statements. The mysql2sqlite script found in the utils directory 
+is used for this conversion. First remove the lines from hpr.sql that 
+mysql2sqlite can't handle:
+
+`sed '/^DELIMITER ;;/,/^DELIMITER ;/d' < ./hpr.sql > ./hpr-sqlite.sql`
+
+Next run the mysql2sqlite script piping its output into the sqlite3 
+program which creates the hpr.db file:
+
+`./utils/mysql2sqlite ./hpr-sqlite.sql | sqlite3 ./hpr.db`
+
+For convenience, the update-sqlite-db.sh script in the utils directory
+automates the above steps (including downloading the hpr.sql file). 
+From the root of the local hpr_generator repository run:
+
+`./utils/update-sqlite-db.sh`
+
+# Configure the site-generator
+
+In your favorite text editor, open the site.cfg file found in the root of the 
+"hpr_generator" folder. Full details about options for configuring the site.cfg 
+file are found in the comments within the file.
+
+## Configuring the database connection
+
+Any database supported by the Perl:DBI and Perl::DBD modules can be used with 
+the site-generator program. Currently the hpr_generator project works with 
+a MySQL or SQLite database.
+
+Find the [DBI] section of the file. It should look like the following
+
+```
+[DBI]
+# Configuration settings for SQLite
+#database: sqlite
+#driver: dbi:SQLite:hpr.db 
+#user:        (not used - leave blank)
+#password:    (not used - leave blank)
+# Configuration settings for MySQL
+#database: mysql
+#driver: dbi:mysql:database=hpr_hpr:hostname=localhost 
+#user: hpr-generator  (Suggested user with read-only privileges)
+#password: *********  (Password for user)
+```
+
+### SQLite
+
+Remove the comment character from the start of the database and driver 
+option lines:
+
+ ```
+# Configuration settings for SQLite
+database: sqlite
+driver: dbi:SQLite:hpr.db 
+#user:        (not used - leave blank)
+#password:    (not used - leave blank)
+```
+
+The hpr.db section of the driver option `dbi:SQLite:hpr.db` is the path 
+to the sqlite file. The default assumes the hpr.db file is located in the same
+directory as the site-generator. 
+
+### MySQL
+
+Remove the comment character from the start of the database, driver, 
+user, and password option lines:
+
+ ```
+# Configuration settings for MySQL
+database: mysql
+driver: dbi:mysql:database=hpr_hpr:hostname=localhost 
+user: hpr-generator
+password: *********
+```
+
+This assumes that the MySQL database service is available at the localhost 
+hostname, that the database name (hpr_hpr) is the database created from 
+the hpr.sql dump file or manually created by you, that the user (hpr-generator) 
+was added by you and has read rights to the hpr_hpr database, and that the 
+password (replace ********* with the actual password) matches the password set 
+for the hpr-generator database user.
+
+## Configuring the website for viewing locally
+
+For HTML links to work when viewing the files on your local machine using the 
+"file://" protocal (i.e. using the "Open..." command in your browser, each HTML 
+file must include a \<base\> meta-data tag in the \<head\> section of its 
+contents. To configure this in the site.cfg file, find the [root_template] 
+section. It should look like the following:
+
+```
+[root_template]
+content: page.tpl.html
+#baseurl: OPTIONAL [i.e. file://<full path to local website directory>]
+```
+Below the #baseurl comment line add:
+
+```
+baseurl: file://</path/to>/hpr_generator/public_html
+```
+
+Replace \<path/to\> with the full path to the hpr_generator directory. For 
+example: `file:///home/rho_n/development/hpr_generator/public_html`
+
+## Configuring the website media file links
+
+If you do not want to host all the media files (currently, audio files and 
+transcription files), you can configure the `media_baseurl` option. This can 
+be added to the [root_template] section of the site.cfg file. Suggested 
+external site is archive.org. To use this site add:
+
+```
+media_baseurl: https://archive.org/download/hpr$eps_id/
+```
+
+# Run the site-generator
+
+Run the site generator form the hpr_generator directory:
+
+```
+./site-generator --all
+```
+
+This will generate all the files for the website. For more examples and to see 
+all available options run:
+
+```
+./site-generator --help
+```

README.md

@@ -4,10 +4,15 @@ Static web page generator for the Hacker Public Radio website.
 ## Installation
 * Clone or download this repository
 * With SQLite
-	* Create the sqlite3 database from the files in the _sql directory. The default name for the database file is "hpr.db" and should be located in the root of the project directory. The name and location can be set in the site.cfg file.
+	* Create the sqlite3 database from the hpr.sql MySQL dump file available on 
-	* Two sql helper scripts are available to generate an empty database or a database filled with test data.
+	  hackerpublicradio.org. The default name for the database file is "hpr.db" 
-		- For an empty database: `cat Create_Database_Empty.sql | sqlite3 hpr.db`
+	  and should be located in the root of the project directory. The name and 
-		- For a database with test data: `cat Create_Database_Test.sql | sqlite3 hpr.db`
+	  location can be set in the site.cfg file.
+	* An "update-hpr.sh" helper script is available in the utils directory. This 
+	  script will download the hpr.sql file, convert it to the SQLite hpr.db file, 
+	  and regenerate the website using the site-generator. 
+		1. `cd` into the root of the project directory
+		2.  Run `./utils/update-hpr.sh`
 	* SQLite v3.8.3 or greater is recommended. CTE WITH clauses are used in some template queries. Must convert WITH 
 	  clauses to sub-queries when using earlier versions of SQLite.
 * With MySQL
@@ -30,7 +35,8 @@ Static web page generator for the Hacker Public Radio website.
     * Tie::DBI
     * DBD::SQLite or DBD:mysql
     * Date::Calc
-
+* See the Getting Started tutorial (GETTING_STARTED.md) for more details on 
+  installing the HPR generator.
 ## Usage
 Generate two specific pages:
 `site-generator index about`		
@@ -60,3 +66,6 @@ and add the label "**Feature Request**".
 
 ## Authors and acknowledgment
 * Roan "Rho`n" Horning 
+* gordons
+* Ken Fallon
+* norrist

public_html/css/hpr.css

@@ -248,6 +248,11 @@ pre.comment {
 	line-height: 1.2rem;
 }
 
+h3.title { margin: 1.25rem 0 0.75rem 0 }
+p.meta { margin: 0.25rem 0 }
+p.summary { margin: 0.25rem 0 }
+p.listen-in { margin-top: 0 }
+
 @media only screen and (max-width: 680px) { 
 	#container {width: 95%;}
 	img#hprlogo {float: left; max-width: 85px; padding: .35em;}

site-generator

@@ -8,12 +8,13 @@
 
 	site-generator [OPTION]... PAGE|PAGE=<comma separated list of ids>...
 
-	-a, --all      generate all pages defined in configuration file
+	-a, --all       generate all pages defined in configuration file
-	-l, --list     print list of configured pages
+	-c, --configure path to configuration file
-	-p, --preview  print generated pages to standard out
+	-l, --list      print list of configured pages
-	-q, --quiet    suppress progress information while generating pages
+	-p, --preview   print generated pages to standard out
-	-v, --verbose  print extended progress information while generating pages
+	-q, --quiet     suppress progress information while generating pages
-	--help         print this help message
+	-v, --verbose   print extended progress information while generating pages
+	--help          print this help message
 
 	Where I<PAGE> is a file name of a web page 
 	or the special I<ALL> (to generate all pages).
@@ -36,12 +37,15 @@ This is a site generator for the Hacker Public Radio website based upon the Perl
 =head1 INSTALLATION
 
 	With SQLite
-	* Create the sqlite3 database from the files in the _sql directory. The default name for the 
+	* Create the sqlite3 database from the hpr.sql MySQL dump file available on 
-	  database file is "hpr.db" and should be located in the root of the project directory. The 
+	  hackerpublicradio.org. The default name for the database file is "hpr.db" 
-	  name and location can be set in the site.cfg file.
+	  and should be located in the root of the project directory. The name and 
-	* Two sql helper scripts are available to generate an empty database or a database filled with test data.
+	  location can be set in the site.cfg file.
-		- For an empty database: cat Create_Database_Empty.sql | sqlite3 hpr.db
+	* An "update-hpr.sh" helper script is available in the utils directory. This 
-		- For a database with test data: cat Create_Database_Test.sql | sqlite3 hpr.db
+	  script will download the hpr.sql file, convert it to the SQLite hpr.db file, 
+	  and regenerate the website using the site-generator. 
+		1. `cd` into the root of the project directory
+		2.  Run `./utils/update-hpr.sh`
 	* SQLite v3.8.3 or greater is recommended. CTE WITH clauses are used in some template queries. 
 	  Must convert WITH clauses to sub-queries when using earlier versions of SQLite.
 
@@ -106,15 +110,17 @@ sub main {
 
 	# Argument parsing
 	my $all;
+	my $configuration_path;
 	my $preview;
 	my $verbose;
 	my $quiet;
 	GetOptions(
-		'all'     => \$all,
+		'all'			=> \$all,
-		'list'    => \&print_available_pages,
+		'configuration=s' => \$configuration_path,
-		'preview' => \$preview,
+		'list'			=> \&print_available_pages,
-		'verbose' => \$verbose,
+		'preview'		=> \$preview,
-		'quiet'   => \$quiet,
+		'verbose'		=> \$verbose,
+		'quiet'			=> \$quiet,
 	) or pod2usage(1);
 	pod2usage(1) unless @ARGV || $all;
 	my (@page_args) = @ARGV;
@@ -123,10 +129,21 @@ sub main {
 		$verbose = 'quiet';
 	};
 
-	# Load config file
+	if (!$configuration_path) {
-	read_config "site.cfg" => my %config;
+		$configuration_path = "site.cfg";
+	}
 
-	my $tt = get_template_html($config{DBI});
+	my %config;
+	if ( -f $configuration_path ) {
+		# Load config file
+		read_config $configuration_path => %config;
+	}
+	else {
+		print STDOUT "Could not read configuration file: $configuration_path\n"; 
+		exit 1;
+	}
+
+	my $tt = get_template_html($config{DBI}, $config{app_paths});
 
 	# If command line option all is set, parse configuration file 
 	# for all pages
@@ -137,7 +154,7 @@ sub main {
 		# from the generated list of pages.
 		@page_args= grep { $_ ne 'DBI' } @page_args;
 		@page_args= grep { $_ ne 'root_template' } @page_args;
-		@page_args= grep { $_ ne 'media_hostname' } @page_args;
+		@page_args= grep { $_ ne 'app_paths' } @page_args;
 
 	};
 	foreach my $page_arg (@page_args) {
@@ -190,8 +207,8 @@ sub get_template_html (\%@)  {
 	# HTML comments to make the template file valid HTML.
 	#
 	return Template->new({
-			INCLUDE_PATH => './templates',
+			INCLUDE_PATH => $_[1]{templates_path},
-			OUTPUT_PATH	 => './public_html',
+			OUTPUT_PATH	 => $_[1]{output_path},
 			EVAL_PERL    => 1,
 			START_TAG	 => '<!--%',
 			END_TAG		 => '%-->',

site.cfg

@@ -2,10 +2,21 @@
 # with the database.
 # dbi:<driver name [SQLite, CSV, ADO, mSQL, etc.]>:<database name>
 [DBI]
-database: mysql
+# Configuration settings for SQLite
-driver: dbi:mysql:database=hpr_hpr:hostname=localhost 
+#database: sqlite
-user: hpr-generator
+#driver: dbi:SQLite:hpr.db 
-password: zBozqN-Z2zNAz
+#user:        (not used - leave blank)
+#password:    (not used - leave blank)
+# Configuration settings for MySQL
+#database: mysql
+#driver: dbi:mysql:database=hpr_hpr:hostname=localhost 
+#user: hpr-generator  (Suggested user with read-only privileges)
+#password: *********  (Password for user)
+
+# Configure the location of the templates and the generated HTML
+[app_paths]
+templates_path: ./templates
+output_path: ./public_html
 
 # Configure the root template page which pulls in the navigation and 
 # content templates used by each page. An optional baseurl property may 
@@ -15,6 +26,7 @@ password: zBozqN-Z2zNAz
 content: page.tpl.html
 #baseurl: OPTIONAL [i.e. file://<full path to local website directory>]
 baseurl: file:///home/roan/Development/hpr/website/hpr_generator/public_html/
+hub_baseurl: http://hub.hackerpublicradio.com/
 media_baseurl: https://archive.org/download/hpr$eps_id/
 
 # Configure the navigation menu and the content templates for each page 
@@ -135,6 +147,10 @@ content: content-promote.tpl.html
 navigation: navigation-about.tpl.html
 content: content-sitemap.tpl.html
 
+[tags]
+navigation: navigation-main.tpl.html
+content: content-tags.tpl.html
+
 [twat_episode]
 navigation: navigation-get-shows.tpl.html
 content: content-twat_episode.tpl.html

templates/content-about.tpl.html

@@ -1,6 +1,6 @@
   <h1>About HPR.</h1>
   <h2 id="history">History<a href="#history">.</a></h2>
-  <p>Hacker Public Radio (HPR) is an Internet Radio show (podcast) that releases shows every weekday Monday through Friday. HPR has a long lineage going back to <a target="_blank" href="http://www.oldskoolphreak.com" target="_blank">Radio FreeK America</a>, <a target="_blank" href="http://www.binrev.com" target="_blank">Binary Revolution Radio</a> &amp; <a target="_blank" href="http://nomicon.info" target="_blank">Infonomicon</a>, and it is a direct continuation of <a target="_blank" href="<!--% absolute_url(baseurl) %-->twat.html" target="_blank">Twatech</a> radio. Please listen to StankDawg&#39;s &quot;<a target="_blank" href="<!--% absolute_url(baseurl) %-->eps/hpr0001.mp3" target="_blank">Introduction to HPR</a>&quot; for more information.</p>
+  <p>Hacker Public Radio (HPR) is an Internet Radio show (podcast) that releases shows every weekday Monday through Friday. HPR has a long lineage going back to <a target="_blank" href="http://www.oldskoolphreak.com" target="_blank">Radio FreeK America</a>, <a target="_blank" href="http://www.binrev.com" target="_blank">Binary Revolution Radio</a> &amp; <a target="_blank" href="http://nomicon.info" target="_blank">Infonomicon</a>, and it is a direct continuation of <a target="_blank" href="<!--% absolute_url(baseurl) %-->twat.html" target="_blank">Twatech</a> radio. Please listen to StankDawg&#39;s &quot;<a target="_blank" href="<!--% media_path(1, 'hpr', 'mp3', baseurl, media_baseurl) %-->" target="_blank">Introduction to HPR</a>&quot; for more information.</p>
 
   <h2 id="community">The HPR Community<a href="#community">.</a></h2>
   <p>What differentiates HPR from other podcasts is that the shows are crowd sourced from the community - fellow listeners like <strong><a target="_blank" href="<!--% absolute_url(baseurl) %-->contact.html" target="_blank">you</a></strong>. There is no restriction on how long the show can be, nor on the topic you can cover as long as they are not spam and <em>&quot;are of interest to <a target="_blank" href="http://en.wikipedia.org/wiki/Hacker_(hobbyist)" target="_blank">Hackers</a>&quot;</em>. If you want to see what topics have been covered so far just have a look at our <a target="_blank" href="<!--% absolute_url(baseurl) %-->eps/index.html" target="_blank">Archive</a>. We also allow for a <a target="_blank" href="<!--% absolute_url(baseurl) %-->series/index.html" target="_blank">series</a> of shows so that hosts can go into more detail on a topic.<p>

templates/content-comments_viewer.tpl.html

@@ -1,5 +1,4 @@
 <!--% PROCESS 'shared-episode-summary.tpl.html' %-->
-<!--% PROCESS 'shared-avatar.tpl.html' %-->
 <!--% PROCESS 'shared-utils.tpl.html' %-->
 <!--% PROCESS 'shared-listen-now.tpl.html' %-->
 <!--% PROCESS "queries-episode-${constants.database}.tpl.html" %-->

templates/content-contribute.tpl.html

@@ -55,7 +55,7 @@
   
   <h2>4. Upload your show</h2>
   <p>
-  Once you have your audio ready you can post your show by going to the <a href="https://www.hackerpublicradio.org/calendar.php">Calendar</a> page or press the <strong><a href="https://www.hackerpublicradio.org/calendar.php">⇧Upload⇧</strong></a> button.
+  Once you have your audio ready you can post your show by going to the <a href="<!--% hub_baseurl %-->calendar.php">Calendar</a> page or press the <strong><a href="<!--% hub_baseurl %-->calendar.php">⇧Upload⇧</strong></a> button.
   </p>
   <p>
   After you have selected a date you will be redirected to the <a href="https://www.hackerpublicradio.org/request.php">request page</a>, where you will be asked to select your show and email address. We will send a link to where you can upload your show to this address and so it must be a valid working address that you can access quickly. We intend to use <a href="https://www.gnupg.org/">GPG</a> to speed up the validation of hosts, so you may wish to use an email address that has an associated public key available on public key servers.

templates/content-correspondent.tpl.html

@@ -55,19 +55,20 @@
 		<!--% this_host = hosts.$id %-->
 		<h2 class="title">Correspondent</h2>
 		<h2><!--% this_host.host %--></h2>
-		<p><img src="<!--% absolute_path(baseurl) %-->images/<!--% get_avatar(this_host.hostid) %-->" 
+		<p><!--% get_avatar(this_host.hostid, this_host.host) %--><br>
-				height="80" alt="Host Image" /><br>
 		<label>Host ID</label>: <!--% this_host.hostid %--><br><br>
 		<label>email:</label> <u><!--% this_host.email %--></u><br>
 		<label>episodes:</label> <strong><!--% hpr_show_count + twat_show_count %--></strong>
 		</p>
 	<!--% FOREACH hpr_show IN hpr_shows; %-->
 
-		<!--% show_summary(hpr_show, 'hide_host') %-->
+	<!--% show_summary(hpr_show, 'hide_host') %-->
-    <!--% END %-->
+    <p class="listen-in"><!--% display_listen_in(hpr_show.id) %--></p>
+	<!--% END %-->
 	<!--% FOREACH twat_show IN twat_shows; %-->
 
 		<!--% twat_show_summary(twat_show, 'hide_host') %-->
+		<p class="listen-in"><!--% display_listen_in(twat_show.id,"twat") %--></p>
     <!--% END %-->
 	</article>
 	<p><a href="contribute.html">Become a Correspondent</a></p>

templates/content-correspondents.tpl.html

@@ -1,9 +1,10 @@
 <!--% PROCESS 'shared-utils.tpl.html' %-->
+<!--% PROCESS 'shared-avatar.tpl.html' %-->
 <article>
 	<h2 class="title">Correspondents</h2>
 	<p>For more information on how to become a Correspondent see our <a href="<!--% absolute_path(baseurl) %-->contribute.html">contribute</a></center> page. To add a logo here, either email one to admin at hpr or setup your email on <a href="https://en.gravatar.com/">Gravatar</a>. To protect your browsing privacy we gather the images every hour and serve them directly from HPR.<p />
 <!--% USE DBI(constants.driver, constants.user, constants.password) %-->
-<!--% PROCESS 'shared-avatar.tpl.html' %-->
+<!--% host_cnt = 0 %-->
 	<table class="hosts">
 	<th >Avatar</th>
 	<th >Name &amp; Host ID</th>
@@ -17,14 +18,14 @@
 		on h.hostid = e.hostid '
 	) %-->
 	<tr height="80">
-	<td><img src="<!--% absolute_path(baseurl) %-->images/<!--% get_avatar(host.hostid) %-->" height="80" width="80" alt="avatar" /></td>
+	<td><!--% get_avatar(host.hostid, host.host, host_cnt > 8) %--></td>
 
 		<td><strong><!--% host.host %--></strong><br>
 
 			Host ID: <a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pad_left(host.hostid) %-->.html"><!--% host.hostid %--></a></td>
 		<td><!--% host.license %--></td>
 		<td><!--% host.date %--></td>
-	</tr>
+	</tr><!--% host_cnt = host_cnt + 1 %-->
 	<!--% END %-->
 	</table>
 	<p><a href="<!--% absolute_path(baseurl) %-->contribute.html">Become a Correspondent</a></p>

templates/content-episode.tpl.html

@@ -16,22 +16,19 @@
 		<h3><!--% episode.summary %--></h3>
 		<p class="meta"><!--% episode_navigation %-->
 		</p>
-<p><!--% IF get_avatar(episode.hostid) != 'hpr_logo.png' %-->
+<p><!--% show_avatar(episode.hostid, episode.host) %-->
-<img src="<!--% absolute_path(baseurl) %-->images/<!--% get_avatar(episode.hostid) %-->" height="80" width="80" alt="" /></a><br><!--% END %-->
 Hosted by <a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pad_left(episode.hostid) %-->.html"><!--% episode.host %--></a> on <span><!--% episode.date %--></span> is flagged as <span><!--% display_explicit(episode.explicit) %--></span> and is released under a <span><!--% episode.license %--> license</span>. <br>
-<!--% display_tags(episode.tags) %--> <br>
+<!--% display_tags(episode.tags) %-->  
-<!--% display_listen_in(episode.id) %--> | 
+
-<!--% display_comments_tally(episode.id, episode.eps_tally) %-->
+<label>Comments: </label><!--% display_comments_tally(episode.id, episode.eps_tally) %--> <br>
+<!--% listen_now(episode, "hpr", baseurl, media_baseurl) %-->
 </p>
 <h3><!--% show_series(episode.series, episode.seriesid, "Part of the series") %--></h3> 
 <p><em><!--% episode.series_description %--></em></p>
 	</header>
 	<div><!--% episode.notes %--></div>
 <footer>
-
 <!--% show_transcript(episode, "hpr", baseurl, media_baseurl) %-->
-
-<!--% listen_now(episode, "hpr", baseurl, media_baseurl) %-->
 <p>
 <!--% episode_navigation %-->
 </p>

templates/content-episodes.tpl.html

@@ -11,6 +11,7 @@
 <!--% FOREACH episodes IN DBI.query(query_episodes) 
 %-->
 	<!--% show_summary(episodes) %-->
+	<p class="listen-in"><!--% display_listen_in(episodes.id) %--></p>
 <!--% END %-->
 <h2 id="twat_episodes">Today With a Techie Archived Shows</h2>
 	<p>

templates/content-index.tpl.html

@@ -5,6 +5,24 @@
 <!--% PROCESS 'shared-call_for_shows.tpl.html' %-->
 <!--% INCLUDE 'content-index-announcement.tpl.html' %-->
 <!--% PROCESS "queries-index-${constants.database}.tpl.html" %-->
+<!--% MACRO tidy_notes(all_lines) BLOCK %-->
+<!--% lines = all_lines %-->
+<!--% after_html = all_lines %-->
+<!--% img_re = '(?six)^(.*?)\<img ([^\>]+\>)(.+)$' %-->
+<!--% lazy_re = '(?i)loading="lazy"' %-->
+<!--% WHILE (matches = lines.match(img_re)) %-->
+<!--% img_tag = matches.1 %-->
+<!--% after_html = matches.2 %-->
+<!--% matches.0 %-->
+<!--% IF (img_tag.search(lazy_re)) %-->
+    <img <!--% img_tag %-->
+<!--% ELSE %-->
+    <img loading="lazy" <!--% img_tag %-->
+<!--% END %-->
+<!--% lines = after_html %-->
+<!--% END %-->
+<!--% after_html %-->
+<!--% END %-->
 <hr>
   <article>
     <header>
@@ -16,11 +34,11 @@
 	%-->
 	<!--% USE date %-->
 	<!--% calc = date.calc %-->
-	<!--%  days_till_next_episode = calc.Delta_Days(date.format(date.now, '%Y'),date.format(date.now, '%m'),date.format(date.now, '%d'),next_available_episode_result.last_year,next_available_episode_result.last_month,next_available_episode_result.last_day + 1) %-->
+	<!--%  days_till_next_episode = calc.Delta_Days(date.format(date.now, '%Y'),date.format(date.now, '%m'),date.format(date.now, '%d'),next_available_episode_result.last_year,next_available_episode_result.last_month,next_available_episode_result.last_day) %-->
 	<!--% END %-->
 	<!--% USE date %-->  
 	<!--% delta = date.calc.N_Delta_YMD(2005,9,19, date.format(date.now, '%Y'),date.format(date.now, '%m'),date.format(date.now, '%d')) %-->
-	<p>We started producing shows as <a href="<!--% absolute_path(baseurl) %-->eps/index.html#twat_episodes"><em>Today with a Techie</em></a> on 2005-09-19, <!--% delta.0 %--> years, <!--% delta.1 %--> months, <!--% delta.2 %--> days ago. our shows are produced by <a href="<!--% absolute_path(baseurl) %-->correspondents/index.html">listeners</a> like you and can be on any <a href="<!--% absolute_path(baseurl) %-->eps/index.html">topic</a> that <strong>"are of interest to <a href="https://en.wikipedia.org/wiki/hacker_(hobbyist)">hackers</a>"</strong>. if you listen to HPR then please consider contributing one show a year. if you <a href="<!--% absolute_path(baseurl) %-->contribute.html">record</a> your show now it could be <a href="https://www.hackerpublicradio.org/calendar.php">released</a> in <strong><!--% days_till_next_episode %--></strong> days.</p>
+	<p>We started producing shows as <a href="<!--% absolute_path(baseurl) %-->eps/index.html#twat_episodes"><em>Today with a Techie</em></a> on 2005-09-19, <!--% delta.0 %--> years, <!--% delta.1 %--> months, <!--% delta.2 %--> days ago. our shows are produced by <a href="<!--% absolute_path(baseurl) %-->correspondents/index.html">listeners</a> like you and can be on any <a href="<!--% absolute_path(baseurl) %-->eps/index.html">topic</a> that <strong>"are of interest to <a href="https://en.wikipedia.org/wiki/hacker_(hobbyist)">hackers</a>"</strong>. if you listen to HPR then please consider contributing one show a year. if you <a href="<!--% absolute_path(baseurl) %-->contribute.html">record</a> your show now it could be <a href="<!--% hub_baseurl %-->calendar.php">released</a> in <strong><!--% days_till_next_episode %--></strong> days.</p>
   </article>
   <!--% display_call_for_shows() %-->
   <article>
@@ -35,6 +53,7 @@
   <hr>
   <!--% PROCESS 'shared-episode-summary.tpl.html' %-->
 <h1>Latest Shows</h1>
+<!--% host_cnt = 0 %-->
 <!--% FOREACH latest_episodes IN DBI.query(query_latest_episodes)
 %-->
 <hr>
@@ -42,21 +61,20 @@
 	<header>
 			<h1><a href="<!--% absolute_path(baseurl) %-->eps/hpr<!--% zero_pad_left(latest_episodes.id) %-->/index.html">hpr<!--% latest_episodes.id %--> :: <!--% latest_episodes.title %--></a></h1>
 		<h3><!--% latest_episodes.summary%--></h3>
-		<p class="meta"$><!--% IF get_avatar(latest_episodes.hostid) != 'hpr_logo.png' %-->
+		<p class="meta"$><!--% show_avatar(latest_episodes.hostid, latest_episodes.host, host_cnt) %-->
-		<img src="<!--% absolute_path(baseurl) %-->images/<!--% get_avatar(latest_episodes.hostid) %-->" height="80" width="80" alt="" /></a><!--% END %-->
+	Hosted by <a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pad_left(latest_episodes.hostid) %-->.html"><!--% latest_episodes.host %--></a> on <!--% latest_episodes.date %--> is flagged as <!--% display_explicit(latest_episodes.explicit) %--> and released under a <!--% latest_episodes.license %--> license. <br>
-	<br>Hosted by <a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pad_left(latest_episodes.hostid) %-->.html"><!--% latest_episodes.host %--></a> on <!--% latest_episodes.date %--> is flagged as <!--% display_explicit(latest_episodes.explicit) %--> and released under a <!--% latest_episodes.license %--> license. <br>
+	<!--% display_tags(latest_episodes.tags) %--> 
-<!--% display_tags(latest_episodes.tags) %--><br>
+	<span><!--% show_series(latest_episodes.series, latest_episodes.seriesid) %--></span>
-<small><!--% display_listen_in(latest_episodes.id) %--> <!--% show_series(latest_episodes.series, latest_episodes.seriesid) %--> | 
+	<span><label>Comments: </label><!--% display_comments_tally(latest_episodes.id, latest_episodes.eps_tally) %--></span><br>
-<!--% display_comments_tally(latest_episodes.id, latest_episodes.eps_tally) %--></small>
+<!--% listen_now(latest_episodes, "hpr", baseurl, media_baseurl) %-->
 		</p>
 	</header>
-		<!--% latest_episodes.notes %-->
+		<!--% tidy_notes(latest_episodes.notes) %-->
 	<footer>
 	<!--% show_transcript(latest_episodes, "hpr", baseurl, media_baseurl) %-->
-
-	<!--% listen_now(latest_episodes, "hpr", baseurl, media_baseurl) %-->
 	</footer>
 </article>
+<!--% host_cnt = host_cnt + 1 %-->
 <!--% END %-->
 <article>
     <header>
@@ -66,6 +84,7 @@
 <!--% FOREACH last_5_weeks_episodes IN DBI.query(query_last_5_weeks_episodes) 
 %-->
 	<!--% show_summary(last_5_weeks_episodes) %-->
+	<p class="listen-in"><!--% display_listen_in(last_5_weeks_episodes.id) %--></p>
 <!--% END %-->
 </article>
 <h1>Older Shows</h1>

templates/content-request_a_slot.tpl.html

@@ -7,7 +7,7 @@
   This is intended only for exceptional circumstances, such as a scheduled interview where we would like the audio to be released as soon after the event as possible, or to cover an important topical situation that has occurred. Due to the extended time now needed to post shows to external sites, the extra work this entails and the disruptive effect of reservations, we will no longer be allowing them except in very rare cases of the type mentioned above. [<a href="http://hackerpublicradio.org/pipermail/hpr_hackerpublicradio.org/2018-February/014260.html">?</a>]
   </p>
   <p>
-  You can post your show by going to the <a href="http://www.hackerpublicradio.org/calendar.php">Calendar</a> page. Once you have selected a date you will be redirected to the <a href="<!--% absolute_url(baseurl) %-->request.html">request page</a>, where you will be asked to select your show and email address. We will send a link to where you can upload your show to this address and therefore it must be a valid working address that you can access quickly. We intend to use <a href="https://www.gnupg.org/">GPG</a> to speed up the validation of hosts, so you may wish to use an email address that has an associated public key available on public key servers. 
+  You can post your show by going to the <a href="<!--% hub_baseurl %-->calendar.php">Calendar</a> page. Once you have selected a date you will be redirected to the <a href="<!--% absolute_url(baseurl) %-->request.html">request page</a>, where you will be asked to select your show and email address. We will send a link to where you can upload your show to this address and therefore it must be a valid working address that you can access quickly. We intend to use <a href="https://www.gnupg.org/">GPG</a> to speed up the validation of hosts, so you may wish to use an email address that has an associated public key available on public key servers. 
   </p>
   <p>
   The audio of your show will not be moderated, however <em>the information provided in the upload form <strong>may be moderated</strong></em>. All publicly facing material is managed by the HPR Community as a whole, and not just the host providing the show. The show <a href="<!--% absolute_url(baseurl) %-->eps/hpr2210/index.html">hpr2210 :: On Freedom of Speech and Censorship</a> describes the agreed approach to this topic.
@@ -134,6 +134,6 @@
     </li>
   </ul>
   <p>
-  Press submit and your show will be sent to the HPR Volunteers to be processed. Keep an eye on the <a href="http://www.hackerpublicradio.org/calendar.php">calendar</a> for when it is posted.
+  Press submit and your show will be sent to the HPR Volunteers to be processed. Keep an eye on the <a href="<!--% hub_baseurl %-->calendar.php">calendar</a> for when it is posted.
   Your show information is used to process the media files, on the website, in the RSS feeds, and on social media sites.
   </p>

templates/content-series_episode.tpl.html

@@ -22,5 +22,6 @@
 <!--% show_results = query_shows.execute(id) %-->
 <!--% FOREACH show IN show_results %-->
 <!--% show_summary(show) %-->
+<p class="listen-in"><!--% display_listen_in(show.id) %--></p>
 <!--% END %-->
 </article>

templates/content-sitemap.tpl.html

@@ -1,7 +1,7 @@
 <!--% PROCESS 'shared-utils.tpl.html' %-->
 <h1>Sitemap</h1>
   <ul>
-    <li><a href="https://www.hackerpublicradio/calendar.php"><strong>⇧Upload⇧</strong></a> ← Upload Your Show</li>
+    <li><a href="<!--% hub_baseurl %-->calendar.php"><strong>⇧Upload⇧</strong></a> ← Upload Your Show</li>
     <li><a href="<!--% absolute_path(baseurl) %-->index.html"><strong>Home</strong></a> ← The HPR Homepage</li>
     <li><a href="<!--% absolute_path(baseurl) %-->syndication.html">Get Shows</a> ← The RSS Syndication Feeds</li>
     <ul>

templates/content-tags.tpl.html

@@ -0,0 +1,85 @@
+<!--% PROCESS 'shared-utils.tpl.html' %-->
+<!--% PROCESS "queries-tags-${constants.database}.tpl.html" %-->
+<!--% USE DBI(constants.driver, constants.user, constants.password) %-->
+<!--% PERL %-->
+$Template::Stash::PRIVATE = undef; # Allow . in tag
+<!--% END %-->
+<!--% uniq_tag_count = 0 %-->
+<!--% comma_re = '(?x)(?:^|,\s*)(?:"((?>[^"]*)(?:""[^"]*)*)"|([^",]*))'; %-->
+<!--# 'Moka5,interview, "computer science"' -->
+<!--% FOREACH episode IN DBI.query(query_tags);
+    ep_id = episode.id;
+    ep_tags = episode.tags;
+    FOREACH tag_str IN ep_tags.split(comma_re);
+        NEXT UNLESS tag_str;
+        NEXT IF tag_str == '';
+        tag = tag_str.lower;
+        first_char = tag.substr(0,1);
+        IF first_char == '.';
+            first_char = '&#x0002E;'; # Unicode .
+        END;
+        IF all_tags.${first_char}.exists(tag);
+            all_tags.${first_char}.${tag}.count = all_tags.${first_char}.${tag}.count + 1;
+            all_tags.${first_char}.${tag}.urls.push(ep_id);
+        ELSE;
+            all_tags.${first_char}.${tag}.count = 1;
+            uniq_tag_count = uniq_tag_count + 1;
+            all_tags.${first_char}.${tag}.urls = [ep_id];
+        END;
+    END;
+END %-->
+<!--% USE date %-->
+<!--% all_first = ['#','&#x0002E;','/','0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'] %-->
+
+<style>
+    .columns3 { columns: 3 auto; list-style-type: none }
+    hr.thin {
+        border: 0;
+        height: 0;
+        border-top: 1px solid rgba(0, 0, 0, 0.1);
+        border-bottom: 1px solid rgba(255, 255, 255, 0.3);
+    }
+    p.ralign { text-align: right }
+</style>
+<article>
+	<a id="TOP"><h1 class="title">Tag summary</h1></a>
+	<h4 class="date">Page generated on <!--% date.format(date.now, format => '%Y-%m-%d at %H:%M:%S UTC', gmt => 1) %--></h4>
+	<p>This section summarises all of the tags currently used throughout the
+	database. The tags are in alphabetical order and each is followed by links to
+	the show numbers where it is used so you can see the context the author used
+	it in. There are currently <!--% uniq_tag_count %--> unique tags in the system.</p>
+	<h4>Alphabetical index</h4>
+	<p>This is an index to the initial letters of the tags below.</p>
+	<ul class="columns3">
+    <!--% FOREACH first_char IN all_first %-->
+        <!--% IF all_tags.${first_char} %-->
+            <li><a href="<!--% absolute_path(baseurl) %-->tags.html#<!--% all_tags.${first_char}.keys.sort.first %-->"><strong><!--% first_char %--></strong></a></li>
+        <!--% END %-->
+    <!--% END %-->
+	</ul>
+	<hr>
+    <!--% FOREACH first_char IN all_first %-->
+        <!--% IF all_tags.${first_char} %-->
+		<p class="ralign"><a href="<!--% absolute_path(baseurl) %-->tags.html#TOP">&UpArrow; Go to index</a></p>
+	        <h3>Tags beginning with '<!--% first_char %-->'</h3>
+	        <ul>
+                <!--% FOREACH tag IN all_tags.${first_char}.keys.sort %-->
+    	        	<li>
+    	        		<a id="<!--% tag %-->"><strong><!--% tag %--></strong></a>: 
+                        <!--% ep_links = [] %-->
+                        <!--% FOREACH ep_id IN all_tags.${first_char}.${tag}.urls.nsort;
+							ep_links.push("<a href=\"${absolute_path(baseurl)}eps/hpr" _ zero_pad_left(ep_id) _ '/index.html"'
+                                ' target="_blank" aria-label="' _ tag _
+                                ' - show ' _ ep_id _ '">' _ ep_id _ '</a>');
+                        END %-->
+                        <!--% ep_links.join(', ') %-->
+    	        	</li>
+                <!--% END %-->
+	        </ul>
+        <!--% END %-->
+    <!--% END %-->
+	<p><a href="#TOP">Go to TOP of page</a></p>
+</article>
+<!--% PERL %-->
+$Template::Stash::PRIVATE = 1;
+<!--% END %-->

templates/content-twat_episode.tpl.html

@@ -52,10 +52,10 @@
 		<h3><!--% episode.summary %--></h3>
 		<p class="meta"><!--% episode_navigation %-->
 		</p>
-<p><!--% IF get_avatar(episode.hostid) != 'hpr_logo.png' %-->
+<p><!--% show_avatar(episode.hostid, episode.host) %-->
-<img src="<!--% absolute_path(baseurl) %-->images/<!--% get_avatar(episode.hostid) %-->" height="80" width="80" alt="" /></a><br><!--% END %-->
 Hosted by <a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pad_left(episode.hostid) %-->.html"><!--% episode.host %--></a> on <span><!--% episode.date %--></span> is flagged as <span><!--% display_explicit(episode.explicit) %--></span> and is released under a <span><!--% episode.license %--> license</span>. <br>
 <!--% display_tags(episode.tags) %--> <br>
+<!--% listen_now(episode, "twat", baseurl, media_baseurl) %-->
 <!--% display_listen_in(episode.id, "twat") %-->  
 </p>
 <h3><!--% show_series(episode.series, episode.seriesid, "Part of the series") %--></h3> 
@@ -63,7 +63,6 @@ Hosted by <a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pa
 	</header>
 	<div><!--% episode.notes %--></div>
 <footer>
-<!--% listen_now(episode, "twat", baseurl, media_baseurl) %-->
 <p>
 <!--% episode_navigation %-->
 </p>

templates/navigation-about.tpl.html

@@ -1,10 +1,10 @@
 <!--% PROCESS 'shared-utils.tpl.html' %-->
 		<ul>
-			<li><a href="https://www.hackerpublicradio.org/calendar.php"><strong>⇧Upload⇧</strong></a></li>
+			<li><a href="<!--% hub_baseurl %-->calendar.php"><strong>⇧Upload⇧</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->index.html"><strong>Home</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->about.html">About »</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->correspondents/index.html">Hosts</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->contact.html">Contact</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->promote.html">Promote HPR</a></li>
-			<li><a href="<!--% absolute_path(baseurl) %-->comments_viewer.php">Show Comments</a></li>
+			<li><a href="<!--% absolute_path(baseurl) %-->comments_viewer.html">Show Comments</a></li>
 		</ul>

templates/navigation-contribute.tpl.html

@@ -1,6 +1,6 @@
 <!--% PROCESS 'shared-utils.tpl.html' %-->
 		<ul>
-			<li><a href="https://www.hackerpublicradio.org/calendar.php"><strong>⇧Upload⇧</strong></a></li>
+			<li><a href="<!--% hub_baseurl %-->"><strong>⇧Upload⇧</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->index.html"><strong>Home</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->help_out.html">Contribute »</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->contribute.html">Submit Show</a></li>

templates/navigation-get-shows.tpl.html

@@ -1,6 +1,6 @@
 <!--% PROCESS 'shared-utils.tpl.html' %-->
 		<ul>
-			<li><a href="https://www.hackerpublicradio.org/calendar.php"><strong>⇧Upload⇧</strong></a></li>
+			<li><a href="<!--% hub_baseurl %-->"><strong>⇧Upload⇧</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->index.html"><strong>Home</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->syndication.html"><strong>Get Shows »</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->eps/index.html">Full Episode Guide</a></li>

templates/navigation-give-shows.tpl.html

@@ -1,6 +1,6 @@
 <!--% PROCESS 'shared-utils.tpl.html' %-->
 		<ul>
-			<li><a href="https://www.hackerpublicradio.org/calendar.php"><strong>⇧Upload⇧</strong></a></li>
+			<li><a href="<!--% hub_baseurl %-->"><strong>⇧Upload⇧</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->index.html"><strong>Home</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->contribute.html">Give Shows »</strong></a>
 			<li><a href="<!--% absolute_path(baseurl) %-->stuff_you_need_to_know.html">Info</a></li>

templates/navigation-main.tpl.html

@@ -1,11 +1,11 @@
 <!--% PROCESS 'shared-utils.tpl.html' %-->
 		<ul>	
-			<li><a href="https://www.hackerpublicradio.org/calendar.php"><strong>⇧Upload⇧</strong></a></li>
+			<li><a href="<!--% hub_baseurl %-->calendar.php"><strong>⇧Upload⇧</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->index.html"><strong>Home »</strong></a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->syndication.html">Get Shows</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->contribute.html">Give Shows</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->help_out.html">Contribute</a></li>
-			<li><a href="http://www.hackerpublicradio.org/tags.php">Tags</a></li>
+			<li><a href="<!--% absolute_path(baseurl) %-->tags.html">Tags</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->about.html">About</a></li>
 			<li><a href="<!--% absolute_path(baseurl) %-->search.html">Search</a></li>
 		</ul>

templates/page.tpl.html

@@ -24,7 +24,7 @@
   <link rel="alternate" type="application/rss+xml" title="Hacker Public Radio Ogg Vorbis RSS" href="<!--% absolute_path(baseurl) %-->hpr_ogg.rss" />
   <link rel="alternate" type="application/rss+xml" title="Hacker Public Radio Speex RSS" href="<!--% absolute_path(baseurl) %-->hpr_spx.rss" />
   <link rel="alternate" type="application/rss+xml" title="Hacker Public Radio MP3 RSS" href="<!--% absolute_path(baseurl) %-->hpr_mp3.rss" />
-  <link rel="alternate" type="application/rss+xml" title="Hacker Public Radio Comments RSS" href="/comments_rss.php" />
+  <link rel="alternate" type="application/rss+xml" title="Hacker Public Radio Comments RSS" href="<!--% hub_baseurl %-->comments_rss.php" />
   <link rel="license" title="cc by 3.0"  href="https://creativecommons.org/licenses/by-sa/3.0/" />
   <link href="<!--% absolute_path(baseurl) %-->css/hpr.css" rel="stylesheet" />
   <!--[if IE]>
@@ -43,7 +43,11 @@
 			<a href="<!--% absolute_path(baseurl) %-->sitemap.html">Site Map</a>
 			- <a href="#maincontent">skip to main content</a>
 			</p>
-		<h1 id="sitename">Hacker <a href="https://www.hackerpublicradio.org/comments_viewer.php">P</a>ublic <a href="<!--% absolute_path(baseurl) %-->syndication.html">R</a>adio</h1>
+			<h1 id="sitename">
+				<a href="<!--% absolute_path(baseurl) %-->correspondents/index.html">H</a>acker
+				<a href="<!--% absolute_path(baseurl) %-->comments_viewer.html">P</a>ublic
+				<a href="<!--% absolute_path(baseurl) %-->syndication.html">R</a>adio
+			</h1>
 		<h2>Your ideas, projects, opinions - podcasted.</h2>
 		<h3>New episodes Monday through Friday.</h3>
 	</div>

templates/queries-index-mysql.tpl.html

@@ -6,9 +6,9 @@
 		)
 		SELECT 
 			MIN(next_id.id_next) AS \'next_id\', 
-			DATE_FORMAT(MIN(last_date), \'%Y\') AS \'last_year\',
+			DATE_FORMAT(DATE_ADD(MIN(last_date), INTERVAL 1 DAY), \'%Y\') AS \'last_year\',
-			DATE_FORMAT(MIN(last_date), \'%d\') AS \'last_day\',
+			DATE_FORMAT(DATE_ADD(MIN(last_date), INTERVAL 1 DAY), \'%d\') AS \'last_day\',
-			DATE_FORMAT(MIN(last_date), \'%m\') AS \'last_month\'
+			DATE_FORMAT(DATE_ADD(MIN(last_date), INTERVAL 1 DAY), \'%m\') AS \'last_month\'
 		FROM next_id
 		LEFT JOIN eps ON next_id.id_next = eps.id 
 		WHERE eps.id IS NULL

templates/queries-index-sqlite.tpl.html

@@ -6,9 +6,9 @@
 		)
 		SELECT 
 			MIN(next_id.id_next) AS \'next_id\', 
-			strftime(\'%Y\', MIN (last_date)) AS \'last_year\',
+			strftime(\'%Y\', DATE (MIN (last_date), \'+1 Days\')) AS \'last_year\',
-			strftime(\'%d\', MIN (last_date)) AS \'last_day\',
+			strftime(\'%d\', DATE (MIN (last_date), \'+1 Days\')) AS \'last_day\',
-			strftime(\'%m\', MIN (last_date)) AS \'last_month\'
+			strftime(\'%m\', DATE (MIN (last_date), \'+1 Days\')) AS \'last_month\'
 		FROM next_id
 		LEFT JOIN eps ON next_id.id_next = eps.id 
 		WHERE eps.id IS NULL

templates/queries-tags-mysql.tpl.html

@@ -0,0 +1 @@
+<!--% query_tags = 'SELECT id, tags FROM eps' %-->

templates/queries-tags-sqlite.tpl.html

@@ -0,0 +1 @@
+<!--% query_tags = 'SELECT id, tags FROM eps' %-->

templates/shared-avatar.tpl.html

@@ -1,5 +1,16 @@
-<!--% MACRO get_avatar(host_id) BLOCK  %-->
+<!--% MACRO lazy_load(is_lazy) BLOCK %-->
+<!--% IF is_lazy %-->loading="lazy"<!--% END %-->
+<!--% END %-->
+<!--% MACRO get_avatar_src(host_id) BLOCK  %-->
 <!--% TRY %-->
 <!--% USE File ("./public_html/images/hosts/${host_id}.png") %-->hosts/<!--% File.name %-->
 <!--% CATCH File %-->hpr_logo.png<!--% END %-->
 <!--% END %-->
+<!--% MACRO get_avatar(host_id, host_name, is_lazy_loaded) BLOCK %-->
+<img src="<!--% absolute_path(baseurl) %-->images/<!--% get_avatar_src(host_id) %-->" height="80" width="80" alt="Thumbnail of <!--% host_name | html %-->" <!--% lazy_load(is_lazy_loaded) %--> />
+<!--% END %-->
+<!--% MACRO show_avatar(host_id, host_name, is_lazy_loaded) BLOCK %-->
+<!--% IF get_avatar_src(host_id) != 'hpr_logo.png' %-->
+<a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pad_left(host_id) %-->.html"><!--% get_avatar(host_id, host_name, is_lazy_loaded) %--></a><br>
+<!--% END %-->
+<!--% END %-->

templates/shared-episode-summary.tpl.html

@@ -19,20 +19,26 @@ from the series <em><a href="<!--% baseurl %-->series/<!--% zero_pad_left(series
 <!--% IF episode_type == "" %--><!--% episode_type = 'hpr' %--><!--% END %-->
  Listen in 
 <!--% IF episode_type == 'hpr' %-->
-<a href="<!--% media_path(eps_id, episode_type, 'ogg', baseurl, media_baseurl) %-->">ogg</a>,
+<a href="<!--% media_path(eps_id, episode_type, 'ogg', baseurl, media_baseurl) %-->" aria-label="Download <!--% episode_type %--><!--% zero_pad_left(eps_id) %--> as">ogg</a>,
-<a href="<!--% media_path(eps_id, episode_type, 'spx', baseurl, media_baseurl) %-->">spx</a>,
+<a href="<!--% media_path(eps_id, episode_type, 'spx', baseurl, media_baseurl) %-->" aria-label="Download <!--% episode_type %--><!--% zero_pad_left(eps_id) %--> as">spx</a>,
 or 
 <!--% END %-->
-<a href="<!--% media_path(eps_id, episode_type, 'mp3', baseurl, media_baseurl) %-->">mp3</a> format.
+<a href="<!--% media_path(eps_id, episode_type, 'mp3', baseurl, media_baseurl) %-->" aria-label="Download <!--% episode_type %--><!--% zero_pad_left(eps_id) %--> as">mp3</a> format.
 <!--% END %-->
 
 <!--% MACRO display_comments_tally(eps_id, tally) BLOCK %-->
-<a href="<!--% absolute_path(baseurl) %-->eps/hpr<!--% zero_pad_left(eps_id) %-->/index.html#comments">Comments (<!--% tally %-->)</a>.
+<a href="<!--% absolute_path(baseurl) %-->eps/hpr<!--% zero_pad_left(eps_id) %-->/index.html#comments" aria-label="Comments for hpr<!--% zero_pad_left(eps_id) %-->"><!--% IF tally > 0 %--><!--% tally %--><!--% ELSE %-->(Be the first)<!--% END %--></a>.
 <!--% END %-->
 
 <!--% MACRO show_meta(show, hide_tags) BLOCK %-->
-	<span><label>Released:</label> <!--% show.date %-->.</span> <span><label>Duration:</label> <!--% display_episode_duration(show.duration) %-->.</span> <span><label>Flag:</label> <!--% display_explicit(show.explicit) %-->.</span> <span><!--% show_series(show.series, show.seriesid) %--></span>  <br>
+	<span><label>Released:</label> <!--% show.date %-->.</span> 
-	<!--% IF hide_tags == "" %--><!--% display_tags(tags) %--><!--% END %-->
+	<span><label>Duration:</label> <!--% display_episode_duration(show.duration) %-->.</span> 
+	<span><label>Flag:</label> <!--% display_explicit(show.explicit) %-->.</span> 
+	<span><!--% show_series(show.series, show.seriesid) %--></span>  <br>
+	<!--% IF hide_tags == "" %--><!--% display_tags(show.tags) %--><!--% END %-->
+	<!--% IF show.comentsTally || show.commentsTally == 0 %-->
+	<span><label>Comments:</label> <!--% display_comments_tally(show.id, show.commentsTally) %--></span>
+	<!--% END %-->
 <!--% END %-->
 
 <!--% MACRO show_summary(show, hide_host) BLOCK %-->
@@ -41,9 +47,8 @@ or
 		hosted by <a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pad_left(show.hostid) %-->.html"><!--% show.host %--></a>
 	<!--% END %-->
 	</h3>
-	<p class="meta"><!--% show_meta(show) %--><br>
+	<p class="meta"><!--% show_meta(show) %--></p>
-    <!--% show.summary %-->
+	<p class="summary"><!--% show.summary %--></p>
-	</p>
 <!--% END %-->
 
 <!--% MACRO twat_show_summary(show, hide_host) BLOCK %-->
@@ -52,8 +57,7 @@ or
 		hosted by <a href="<!--% absolute_path(baseurl) %-->correspondents/<!--% zero_pad_left(show.hostid) %-->.html"><!--% show.host %--></a>
 	<!--% END %-->
 	</h3>
-	<p class="meta"><!--% show_meta(show) %--><br>
+	<p class="meta"><!--% show_meta(show) %--></p>
-    <!--% show.summary %-->
+	<p class="summary"><!--% show.summary %--></p>
-	</p>
 <!--% END %-->
 

templates/shared-listen-now.tpl.html

@@ -1,16 +1,12 @@
-<!--% MACRO listen_now(episode, episode_type, baseurl, media_baseurl) BLOCK %-->
+<!--% PROCESS 'shared-episode-summary.tpl.html' %-->
-	<h2>Listen Now</h2>
+
-	<p>Duration: <!--% display_episode_duration(episode.duration) %--></p>
+<!--% MACRO listen_now(episode, episode_type, baseurl, media_baseurl, hide_duration) BLOCK %-->
+<p><!--% display_listen_in(episode.id, episode_type) %--> Play now:<br>	
 	<audio controls preload="none">
 	<source src="<!--% media_path(episode.id, episode_type, 'ogg', baseurl, media_baseurl) %-->" type="audio/ogg" >
 		<source src="<!--% media_path(episode.id, episode_type, 'mp3', baseurl, media_baseurl) %-->" type="audio/mpeg" >
 	</audio>
-	<ul>
+	<!--% IF !hide_duration %--><br>
-<!--% IF episode_type != "twat" %-->
+	Duration: <!--% display_episode_duration(episode.duration) %--><!--% END %--></p>
-		<li>ogg: <a href="<!--% media_path(episode.id, episode_type, 'ogg', baseurl, media_baseurl) %-->"><!--% media_path(episode.id, episode_type, 'ogg', baseurl, media_baseurl) %--></a></li>
-		<li>spx: <a href="<!--% media_path(episode.id, episode_type, 'spx', baseurl, media_baseurl) %-->"><!--% media_path(episode.id, episode_type, 'spx', baseurl, media_baseurl) %--></a></li>
-<!--% END %-->
-		<li>mp3: <a href="<!--% media_path(episode.id, episode_type, 'mp3', baseurl, media_baseurl) %-->"><!--% media_path(episode.id, episode_type, 'mp3', baseurl, media_baseurl) %--></a></li>
-	</ul>
 <!--% END %-->
 

utils/lib_utils.sh

@@ -0,0 +1,184 @@
+#!/bin/bash - 
+#===============================================================================
+#
+#          FILE: lib_utils.sh
+# 
+#         USAGE: ./lib_utils.sh 
+# 
+#   DESCRIPTION: functions for scripts used to update local HPR installations 
+#				 using the HPR static site generator
+# 
+#       OPTIONS: ---
+#  REQUIREMENTS: mysql2sqlite (https://github.com/dumblob/mysql2sqlite)
+#          BUGS: ---
+#         NOTES: ---
+#        AUTHOR: Roan "Rho`n" Horning <roan.horning@gmail.com>
+#       CREATED: 02/26/2023 03:27:08 PM -5 UTC
+#      REVISION: ---
+#       LICENSE: GNU AGPLv3 
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU Affero General Public License as published by
+#    the Free Software Foundation, either version 3 of the License, or
+#    (at your option) any later version.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU Affero General Public License for more details.
+#
+#    You should have received a copy of the GNU Affero General Public License
+#    along with this program.  If not, see <https://www.gnu.org/licenses/>.	
+#
+#===============================================================================
+
+set -o nounset                              # Treat unset variables as an error
+
+#---  FUNCTION  ----------------------------------------------------------------
+#          NAME:  make_working_dir
+#   DESCRIPTION:  Creates a local temporary working directory
+#				  SEE: https://stackoverflow.com/questions/4632028/how-to-create-a-temporary-directory#answer-34676160
+#    PARAMETERS:  
+#       RETURNS: The path to the working directory 
+#-------------------------------------------------------------------------------
+function make_working_dir { 
+
+	# the directory of the script
+	local DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+	# the temp directory used, within $DIR
+	# omit the -p parameter to create a temporal directory in 
+	# the default location
+	
+	local WORK_DIR=`mktemp -d -p "$DIR"`
+
+	# check if tmp dir was created
+	if [[ ! "$WORK_DIR" || ! -d "$WORK_DIR" ]]; then
+		echo "Could not create temp dir"
+		exit 1
+	fi
+	echo $WORK_DIR
+}
+
+#---  FUNCTION  ----------------------------------------------------------------
+#          NAME:  clean_working_dir
+#   DESCRIPTION:  Remove local temporary working directory 
+#    PARAMETERS: WORK_DIR -- Temporay directory to be delted
+#       RETURNS:  
+#-------------------------------------------------------------------------------
+function clean_working_dir {
+
+	if [[ -d $1  ]] && expr $1 : '.*/tmp.*' ; then
+		rm -rf $1
+		echo "Deleted temp working directory $1" 
+	else
+		echo "Did not delete directory: $1"
+		echo "Not a temporary directory."
+	fi
+	
+}
+
+#---  FUNCTION  ----------------------------------------------------------------
+#          NAME:  download_hpr_sql
+#   DESCRIPTION:  Download the HPR SQL dump file into a working directory 
+#    PARAMETERS:  
+#       RETURNS:  
+#-------------------------------------------------------------------------------
+function download_hpr_sql { 
+
+	if [[ ! -d $1  ]] || ! expr $1 : '.*/tmp.*' ; 
+	then
+		echo "Please provide the temporary directory when calling this function"
+		return 1
+	fi
+
+	local CURL=`which curl`
+	local WGET=`which wget`
+	local HPR_URL=https://www.hackerpublicradio.org/hpr.sql
+
+	if [ -f $1/hpr.sql ];
+	then
+		echo "Removing temporary hpr.sql"
+		rm $1/hpr.sql
+	else
+		echo "No temporary hpr.sql to remove"
+	fi
+
+	if [ "$CURL" != "" ]; 
+	then
+		curl $HPR_URL --output $1/hpr.sql
+		echo "Downloaded hpr.sql via curl"
+	elif [ "$WGET" != "" ];
+	then
+		wget --directory-prefix=$1 $HPR_URL
+		echo "Downloaded hpr.sql via wget"
+	else
+		echo "Could not download file. Please install either curl or wget."
+		return 1
+	fi
+}
+
+#---  FUNCTION  ----------------------------------------------------------------
+#          NAME:  make_hpr_sqlite_db
+#   DESCRIPTION:  Converts the hpr sql file into an sqlite db file
+#    PARAMETERS:  
+#       RETURNS:  
+#-------------------------------------------------------------------------------
+function make_hpr_sqlite_db { 
+
+	if [[ ! -d $1  ]] || ! expr $1 : '.*/tmp.*' ; 
+	then
+		echo "Please provide the temporary directory when calling this function"
+		return 1
+	fi
+
+	local MYSQL2SQLITE=`which mysql2sqlite`
+	local BIN_PATH=""
+
+	if [ "$MYSQL2SQLITE" = "" ];
+	then
+		if [ $# -gt 1 ] && [ -z "$2" ];
+		then
+			BIN_PATH=$2
+		else
+			BIN_PATH=`find . -type f -name "mysql2sqlite" -print | head -1`
+			if [ "$BIN_PATH" != "" ]; 
+			then
+				BIN_PATH=${BIN_PATH/mysql2sqlite//}
+			else
+				echo "Could not find mysql2sqlite script."
+				return 1
+			fi
+		fi
+	fi
+	echo $BIN_PATH
+
+	if [ -f $1/hpr.db ];
+	then
+		rm $1/hpr.db
+	fi
+
+	# Remove lines from hpr.sql that mysql2sqlite can't handle
+	sed '/^DELIMITER ;;/,/^DELIMITER ;/d' < $1/hpr.sql > $1/hpr-sqlite.sql
+
+	${BIN_PATH}mysql2sqlite $1/hpr-sqlite.sql | sqlite3 $1/hpr.db
+	echo "Created hpr.db"
+}
+
+#---  FUNCTION  ----------------------------------------------------------------
+#          NAME:  copy_to_public_dir
+#   DESCRIPTION: Move HPR sql and db files to public website folder 
+#    PARAMETERS:  
+#       RETURNS:  
+#-------------------------------------------------------------------------------
+function copy_to_public_dir { 
+
+	if [ $# -gt 1 ] && [ ! -z "$1" ] && [ ! -z "$2" ];
+	then
+		cp $1/hpr.sql $2/hpr.sql
+		cp $1/hpr.db $2/hpr.db
+		return 0
+	else 
+		echo "Bad arguments. Can't copy files to public directory."
+	fi
+}

utils/mysql2sqlite

@@ -0,0 +1,289 @@
+#!/usr/bin/awk -f
+
+# Authors: @esperlu, @artemyk, @gkuenning, @dumblob
+
+# FIXME detect empty input file and issue a warning
+
+function printerr( s ){ print s | "cat >&2" }
+
+BEGIN {
+  if( ARGC != 2 ){
+    printerr( \
+      "USAGE:\n"\
+      "       mysql2sqlite dump_mysql.sql > dump_sqlite3.sql\n" \
+      "       OR\n" \
+      "       mysql2sqlite dump_mysql.sql | sqlite3 sqlite.db\n" \
+      "\n" \
+      "NOTES:\n" \
+      "       Dash in filename is not supported, because dash (-) means stdin." )
+    no_END = 1
+    exit 1
+  }
+
+  # Find INT_MAX supported by both this AWK (usually an ISO C signed int)
+  #   and SQlite.
+  # On non-8bit-based architectures, the additional bits are safely ignored.
+
+  # 8bit (lower precision should not exist)
+  s="127"
+  # "63" + 0 avoids potential parser misbehavior
+  if( (s + 0) "" == s ){ INT_MAX_HALF = "63" + 0 }
+  # 16bit
+  s="32767"
+  if( (s + 0) "" == s ){ INT_MAX_HALF = "16383" + 0 }
+  # 32bit
+  s="2147483647"
+  if( (s + 0) "" == s ){ INT_MAX_HALF = "1073741823" + 0 }
+  # 64bit (as INTEGER in SQlite3)
+  s="9223372036854775807"
+  if( (s + 0) "" == s ){ INT_MAX_HALF = "4611686018427387904" + 0 }
+#  # 128bit
+#  s="170141183460469231731687303715884105728"
+#  if( (s + 0) "" == s ){ INT_MAX_HALF = "85070591730234615865843651857942052864" + 0 }
+#  # 256bit
+#  s="57896044618658097711785492504343953926634992332820282019728792003956564819968"
+#  if( (s + 0) "" == s ){ INT_MAX_HALF = "28948022309329048855892746252171976963317496166410141009864396001978282409984" + 0 }
+#  # 512bit
+#  s="6703903964971298549787012499102923063739682910296196688861780721860882015036773488400937149083451713845015929093243025426876941405973284973216824503042048"
+#  if( (s + 0) "" == s ){ INT_MAX_HALF = "3351951982485649274893506249551461531869841455148098344430890360930441007518386744200468574541725856922507964546621512713438470702986642486608412251521024" + 0 }
+#  # 1024bit
+#  s="89884656743115795386465259539451236680898848947115328636715040578866337902750481566354238661203768010560056939935696678829394884407208311246423715319737062188883946712432742638151109800623047059726541476042502884419075341171231440736956555270413618581675255342293149119973622969239858152417678164812112068608"
+#  if( (s + 0) "" == s ){ INT_MAX_HALF = "44942328371557897693232629769725618340449424473557664318357520289433168951375240783177119330601884005280028469967848339414697442203604155623211857659868531094441973356216371319075554900311523529863270738021251442209537670585615720368478277635206809290837627671146574559986811484619929076208839082406056034304" + 0 }
+#  # higher precision probably not needed
+
+  FS=",$"
+  print "PRAGMA synchronous = OFF;"
+  print "PRAGMA journal_mode = MEMORY;"
+  print "BEGIN TRANSACTION;"
+}
+
+# historically 3 spaces separate non-argument local variables
+function bit_to_int( str_bit,   powtwo, i, res, bit, overflow ){
+  powtwo = 1
+  overflow = 0
+  # 011101 = 1*2^0 + 0*2^1 + 1*2^2 ...
+  for( i = length( str_bit ); i > 0; --i ){
+    bit = substr( str_bit, i, 1 )
+    if( overflow || ( bit == 1 && res > INT_MAX_HALF ) ){
+      printerr( \
+        NR ": WARN Bit field overflow, number truncated (LSBs saved, MSBs ignored)." )
+      break
+    }
+    res = res + bit * powtwo
+    # no warning here as it might be the last iteration
+    if( powtwo > INT_MAX_HALF ){ overflow = 1; continue }
+    powtwo = powtwo * 2
+  }
+  return res
+}
+
+# CREATE TRIGGER statements have funny commenting. Remember we are in trigger.
+/^\/\*.*(CREATE.*TRIGGER|create.*trigger)/ {
+  gsub( /^.*(TRIGGER|trigger)/, "CREATE TRIGGER" )
+  print
+  inTrigger = 1
+  next
+}
+# The end of CREATE TRIGGER has a stray comment terminator
+/(END|end) \*\/;;/ { gsub( /\*\//, "" ); print; inTrigger = 0; next }
+# The rest of triggers just get passed through
+inTrigger != 0 { print; next }
+
+# CREATE VIEW looks like a TABLE in comments
+/^\/\*.*(CREATE.*TABLE|create.*table)/ {
+  inView = 1
+  next
+}
+# end of CREATE VIEW
+/^(\).*(ENGINE|engine).*\*\/;)/ {
+  inView = 0
+  next
+}
+# content of CREATE VIEW
+inView != 0 { next }
+
+# skip comments
+/^\/\*/ { next }
+
+# skip PARTITION statements
+/^ *[(]?(PARTITION|partition) +[^ ]+/ { next }
+
+# print all INSERT lines
+( /^ *\(/ && /\) *[,;] *$/ ) || /^(INSERT|insert|REPLACE|replace)/ {
+  prev = ""
+
+  # first replace \\ by \_ that mysqldump never generates to deal with
+  # sequnces like \\n that should be translated into \n, not \<LF>.
+  # After we convert all escapes we replace \_ by backslashes.
+  gsub( /\\\\/, "\\_" )
+
+  # single quotes are escaped by another single quote
+  gsub( /\\'/, "''" )
+  gsub( /\\n/, "\n" )
+  gsub( /\\r/, "\r" )
+  gsub( /\\"/, "\"" )
+  gsub( /\\\032/, "\032" )  # substitute char
+
+  gsub( /\\_/, "\\" )
+
+  # sqlite3 is limited to 16 significant digits of precision
+  while( match( $0, /0x[0-9a-fA-F]{17}/ ) ){
+    hexIssue = 1
+    sub( /0x[0-9a-fA-F]+/, substr( $0, RSTART, RLENGTH-1 ), $0 )
+  }
+  if( hexIssue ){
+    printerr( \
+      NR ": WARN Hex number trimmed (length longer than 16 chars)." )
+    hexIssue = 0
+  }
+  print
+  next
+}
+
+# CREATE DATABASE is not supported
+/^(CREATE DATABASE|create database)/ { next }
+
+# print the CREATE line as is and capture the table name
+/^(CREATE|create)/ {
+  if( $0 ~ /IF NOT EXISTS|if not exists/ || $0 ~ /TEMPORARY|temporary/ ){
+    caseIssue = 1
+    printerr( \
+      NR ": WARN Potential case sensitivity issues with table/column naming\n" \
+      "          (see INFO at the end)." )
+  }
+  if( match( $0, /`[^`]+/ ) ){
+    tableName = substr( $0, RSTART+1, RLENGTH-1 )
+  }
+  aInc = 0
+  prev = ""
+  firstInTable = 1
+  print
+  next
+}
+
+# Replace `FULLTEXT KEY` (probably other `XXXXX KEY`)
+/^  (FULLTEXT KEY|fulltext key)/ { gsub( /[A-Za-z ]+(KEY|key)/, "  KEY" ) }
+
+# Get rid of field lengths in KEY lines
+/ (PRIMARY |primary )?(KEY|key)/ { gsub( /\([0-9]+\)/, "" ) }
+
+aInc == 1 && /PRIMARY KEY|primary key/ { next }
+
+# Replace COLLATE xxx_xxxx_xx statements with COLLATE BINARY
+/ (COLLATE|collate) [a-z0-9_]*/ { gsub( /(COLLATE|collate) [a-z0-9_]*/, "COLLATE BINARY" ) }
+
+# Print all fields definition lines except the `KEY` lines.
+/^  / && !/^(  (KEY|key)|\);)/ {
+  if( match( $0, /[^"`]AUTO_INCREMENT|auto_increment[^"`]/) ){
+    aInc = 1
+    gsub( /AUTO_INCREMENT|auto_increment/, "PRIMARY KEY AUTOINCREMENT" )
+  }
+  gsub( /(UNIQUE KEY|unique key) (`.*`|".*") /, "UNIQUE " )
+  gsub( /(CHARACTER SET|character set) [^ ]+[ ,]/, "" )
+  # FIXME
+  #   CREATE TRIGGER [UpdateLastTime]
+  #   AFTER UPDATE
+  #   ON Package
+  #   FOR EACH ROW
+  #   BEGIN
+  #   UPDATE Package SET LastUpdate = CURRENT_TIMESTAMP WHERE ActionId = old.ActionId;
+  #   END
+  gsub( /(ON|on) (UPDATE|update) (CURRENT_TIMESTAMP|current_timestamp)(\(\))?/, "" )
+  gsub( /(DEFAULT|default) (CURRENT_TIMESTAMP|current_timestamp)(\(\))?/, "DEFAULT current_timestamp")
+  gsub( /(COLLATE|collate) [^ ]+ /, "" )
+  gsub( /(ENUM|enum)[^)]+\)/, "text " )
+  gsub( /(SET|set)\([^)]+\)/, "text " )
+  gsub( /UNSIGNED|unsigned/, "" )
+  gsub( /_utf8mb3/, "" )
+  gsub( /` [^ ]*(INT|int|BIT|bit)[^ ]*/, "` integer" )
+  gsub( /" [^ ]*(INT|int|BIT|bit)[^ ]*/, "\" integer" )
+  ere_bit_field = "[bB]'[10]+'"
+  if( match($0, ere_bit_field) ){
+    sub( ere_bit_field, bit_to_int( substr( $0, RSTART +2, RLENGTH -2 -1 ) ) )
+  }
+
+  # remove USING BTREE and other suffixes for USING, for example: "UNIQUE KEY
+  # `hostname_domain` (`hostname`,`domain`) USING BTREE,"
+  gsub( / USING [^, ]+/, "" )
+
+  # field comments are not supported
+  gsub( / (COMMENT|comment).+$/, "" )
+  # Get commas off end of line
+  gsub( /,.?$/, "" )
+  if( prev ){
+    if( firstInTable ){
+      print prev
+      firstInTable = 0
+    }
+    else {
+      print "," prev
+    }
+  }
+  else {
+    # FIXME check if this is correct in all cases
+    if( match( $1,
+        /(CONSTRAINT|constraint) ["].*["] (FOREIGN KEY|foreign key)/ ) ){
+      print ","
+    }
+  }
+  prev = $1
+}
+
+/ ENGINE| engine/ {
+  if( prev ){
+    if( firstInTable ){
+      print prev
+      firstInTable = 0
+    }
+    else {
+      print "," prev
+    }
+  }
+  prev=""
+  print ");"
+  next
+}
+# `KEY` lines are extracted from the `CREATE` block and stored in array for later print
+# in a separate `CREATE KEY` command. The index name is prefixed by the table name to
+# avoid a sqlite error for duplicate index name.
+/^(  (KEY|key)|\);)/ {
+  if( prev ){
+    if( firstInTable ){
+      print prev
+      firstInTable = 0
+    }
+    else {
+      print "," prev
+    }
+  }
+  prev = ""
+  if( $0 == ");" ){
+    print
+  }
+  else {
+    if( match( $0, /`[^`]+/ ) ){
+      indexName = substr( $0, RSTART+1, RLENGTH-1 )
+    }
+    if( match( $0, /\([^()]+/ ) ){
+      indexKey = substr( $0, RSTART+1, RLENGTH-1 )
+    }
+    # idx_ prefix to avoid name clashes (they really happen!)
+    key[tableName] = key[tableName] "CREATE INDEX \"idx_" \
+       tableName "_" indexName "\" ON \"" tableName "\" (" indexKey ");\n"
+  }
+}
+
+END {
+  if( no_END ){ exit 1}
+  # print all KEY creation lines.
+  for( table in key ){ printf key[table] }
+
+  print "END TRANSACTION;"
+
+  if( caseIssue ){
+    printerr( \
+      "INFO Pure sqlite identifiers are case insensitive (even if quoted\n" \
+      "     or if ASCII) and doesnt cross-check TABLE and TEMPORARY TABLE\n" \
+      "     identifiers. Thus expect errors like \"table T has no column named F\".")
+  }
+}

utils/update-hpr-db.sh

@@ -0,0 +1,54 @@
+#!/bin/bash - 
+#===============================================================================
+#
+#          FILE: update-hpr-db.sh
+# 
+#         USAGE: ./update-hpr-db.sh 
+# 
+#   DESCRIPTION: Download MySQL hpr.sql file and create SQLite3 file
+# 
+#       OPTIONS: ---
+#  REQUIREMENTS: lib_utils.sh
+#          BUGS: ---
+#         NOTES: ---
+#        AUTHOR: Roan "Rho`n" Horning (roan.horning@gmail.com)
+#  ORGANIZATION: 
+#       CREATED: 03/05/2023 07:21:29 PM
+#      REVISION:  ---
+#       LICENSE: GNU AGPLv3 
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU Affero General Public License as published by
+#    the Free Software Foundation, either version 3 of the License, or
+#    (at your option) any later version.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU Affero General Public License for more details.
+#
+#    You should have received a copy of the GNU Affero General Public License
+#    along with this program.  If not, see <https://www.gnu.org/licenses/>.	
+#
+#===============================================================================
+
+set -o nounset                              # Treat unset variables as an error
+
+BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+#
+# Load library functions
+#
+LIB="$BASEDIR/lib_utils.sh"
+[ -e $LIB ] || { echo "Unable to load functions.\n$LIB not found."; exit; }
+source $LIB
+
+WORKING_DIR=`make_working_dir`
+
+download_hpr_sql $WORKING_DIR
+
+make_hpr_sqlite_db $WORKING_DIR
+
+copy_to_public_dir $WORKING_DIR `pwd`
+
+clean_working_dir $WORKING_DIR

utils/update-hpr.sh

@@ -0,0 +1,72 @@
+#!/bin/bash - 
+#===============================================================================
+#
+#          FILE: update-hpr.sh
+# 
+#         USAGE: ./update-hpr.sh 
+# 
+#   DESCRIPTION: Script to update local statically generated HPR website
+# 
+#       OPTIONS: ---
+#  REQUIREMENTS: lib_utils.sh
+#          BUGS: ---
+#         NOTES: ---
+#        AUTHOR: Roan "Rho`n" Horning (roan.horning@gmail.com), 
+#  ORGANIZATION: 
+#       CREATED: 03/03/2023 09:26:29 PM
+#      REVISION:  ---
+#       LICENSE: GNU AGPLv3 
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU Affero General Public License as published by
+#    the Free Software Foundation, either version 3 of the License, or
+#    (at your option) any later version.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU Affero General Public License for more details.
+#
+#    You should have received a copy of the GNU Affero General Public License
+#    along with this program.  If not, see <https://www.gnu.org/licenses/>.	
+#
+#===============================================================================
+
+set -o nounset                              # Treat unset variables as an error
+
+BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
+#
+# Load library functions
+#
+LIB="$BASEDIR/lib_utils.sh"
+[ -e $LIB ] || { echo "Unable to load functions.\n$LIB not found."; exit; }
+source $LIB
+
+WORKING_DIR=`make_working_dir`
+
+download_hpr_sql $WORKING_DIR
+
+make_hpr_sqlite_db $WORKING_DIR
+
+copy_to_public_dir $WORKING_DIR `pwd`
+
+mv hpr.sql public_html/
+
+echo "Update static HTML files"
+
+# Clean up previously generated files
+rm -rf public_html/*.html public_html/correspondents public_html/eps public_html/series
+
+git restore public_html/will-my-show-be-of-interest-to-hackers.html
+
+# stash changes to configuration file to preserve DB settings
+git stash
+
+git pull
+
+git stash pop
+
+./site-generator --quiet --all 
+
+clean_working_dir $WORKING_DIR