Added local anchor links for each individual comment

Reviewed-on: rho_n/hpr_generator#196

GETTING_STARTED.md

@@ -0,0 +1,206 @@
+# Clone the repository
+
+If git is not installed on the operating system, please install it now 
+(see the git documentation for [instructions on installing git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)).
+
+To retrieve the code from the repository on anhonesthost.net (a username and 
+password are required), run:
+
+`git clone https://repo.anhonesthost.net/rho_n/hpr_generator.git`
+
+To retrieve the code from the repository on gitlab.com, run: 
+
+`git clone https://gitlab.com/roan.horning/hpr_generator.git`
+
+On success, an "hpr_generator" directory will be created in the folder from 
+which the clone command was executed containing a local copy of the git repository.
+
+# Install required Perl modules
+
+Installing the Perl modules is the most finicky part of the installation process. 
+The needed Perl modules can be found using the operating system's package 
+manager or using the modules found in the 
+[Comprehensive Perl Archive Network (CPAN)](https://www.cpan.org/).
+
+## Installing modules on Debian based Linux distributions
+
+Run command (tested on Debian 11):
+
+```
+apt install libconfig-std-perl \
+	libtemplate-perl \
+	libtemplate-plugin-dbi-perl \
+	libdbd-sqlite3-perl libdate-calc-perl \
+	libtie-dbi-perl \
+	libtext-csv-xs-perl \
+	libhtml-parser-perl \
+	libtemplate-plugin-html-strip-perl
+
+```
+
+## Using CPAN to install the modules
+
+A cross platform method to install the needed modules is the Perl CPAN application. 
+Make sure both the [make](https://www.gnu.org/software/make/manual/make.html) 
+command and the [cpan](https://perldoc.perl.org/CPAN) command are available. 
+Install them using the operating system's package manager, or from source.
+
+Run commands:
+
+```
+cpan Config::Std
+cpan Template
+cpan Template::Plugin::DBI
+cpan DBD::SQLite
+cpan Date::Calc
+cpan Tie::DBI
+```
+
+# Create the HPR database
+
+The hpr_generator relies on information from a database to generate many of the 
+files for the website (for example: index.html, series/index.html, 
+hpr_mp3.rss, etc). This data is available from a MySQL dump file found on 
+hackerpublicradio.org at URL "https://www.hackerpublicradio.org/hpr.sql".
+
+The first step is to download the hpr.sql file. This can be done using your 
+browser, or by running one of the following commands:
+
+`curl https://www.hackerpublicradio.org/hpr.sql --output ./hpr.sql`
+
+or
+
+`wget --directory-prefix=./ https://www.hackerpublicradio.org/hpr.sql`
+
+## Creating an SQLite database file
+
+The SQL of the hpr.sql file must be converted from MySQL specific statements to 
+SQLite specific statements. The mysql2sqlite script found in the utils directory 
+is used for this conversion. First remove the lines from hpr.sql that 
+mysql2sqlite can't handle:
+
+`sed '/^DELIMITER ;;/,/^DELIMITER ;/d' < ./hpr.sql > ./hpr-sqlite.sql`
+
+Next run the mysql2sqlite script piping its output into the sqlite3 
+program which creates the hpr.db file:
+
+`./utils/mysql2sqlite ./hpr-sqlite.sql | sqlite3 ./hpr.db`
+
+For convenience, the update-sqlite-db.sh script in the utils directory
+automates the above steps (including downloading the hpr.sql file). 
+From the root of the local hpr_generator repository run:
+
+`./utils/update-sqlite-db.sh`
+
+# Configure the site-generator
+
+In your favorite text editor, open the site.cfg file found in the root of the 
+"hpr_generator" folder. Full details about options for configuring the site.cfg 
+file are found in the comments within the file.
+
+## Configuring the database connection
+
+Any database supported by the Perl:DBI and Perl::DBD modules can be used with 
+the site-generator program. Currently the hpr_generator project works with 
+a MySQL or SQLite database.
+
+Find the [DBI] section of the file. It should look like the following
+
+```
+[DBI]
+# Configuration settings for SQLite
+#database: sqlite
+#driver: dbi:SQLite:hpr.db 
+#user:        (not used - leave blank)
+#password:    (not used - leave blank)
+# Configuration settings for MySQL
+#database: mysql
+#driver: dbi:mysql:database=hpr_hpr:hostname=localhost 
+#user: hpr-generator  (Suggested user with read-only privileges)
+#password: *********  (Password for user)
+```
+
+### SQLite
+
+Remove the comment character from the start of the database and driver 
+option lines:
+
+ ```
+# Configuration settings for SQLite
+database: sqlite
+driver: dbi:SQLite:hpr.db 
+#user:        (not used - leave blank)
+#password:    (not used - leave blank)
+```
+
+The hpr.db section of the driver option `dbi:SQLite:hpr.db` is the path 
+to the sqlite file. The default assumes the hpr.db file is located in the same
+directory as the site-generator. 
+
+### MySQL
+
+Remove the comment character from the start of the database, driver, 
+user, and password option lines:
+
+ ```
+# Configuration settings for MySQL
+database: mysql
+driver: dbi:mysql:database=hpr_hpr:hostname=localhost 
+user: hpr-generator
+password: *********
+```
+
+This assumes that the MySQL database service is available at the localhost 
+hostname, that the database name (hpr_hpr) is the database created from 
+the hpr.sql dump file or manually created by you, that the user (hpr-generator) 
+was added by you and has read rights to the hpr_hpr database, and that the 
+password (replace ********* with the actual password) matches the password set 
+for the hpr-generator database user.
+
+## Configuring the website for viewing locally
+
+For HTML links to work when viewing the files on your local machine using the 
+"file://" protocal (i.e. using the "Open..." command in your browser, each HTML 
+file must include a \<base\> meta-data tag in the \<head\> section of its 
+contents. To configure this in the site.cfg file, find the [root_template] 
+section. It should look like the following:
+
+```
+[root_template]
+content: page.tpl.html
+#baseurl: OPTIONAL [i.e. file://<full path to local website directory>]
+```
+Below the #baseurl comment line add:
+
+```
+baseurl: file://</path/to>/hpr_generator/public_html
+```
+
+Replace \<path/to\> with the full path to the hpr_generator directory. For 
+example: `file:///home/rho_n/development/hpr_generator/public_html`
+
+## Configuring the website media file links
+
+If you do not want to host all the media files (currently, audio files and 
+transcription files), you can configure the `media_baseurl` option. This can 
+be added to the [root_template] section of the site.cfg file. Suggested 
+external site is archive.org. To use this site add:
+
+```
+media_baseurl: https://archive.org/download/hpr$eps_id/
+```
+
+# Run the site-generator
+
+Run the site generator form the hpr_generator directory:
+
+```
+./site-generator --all
+```
+
+This will generate all the files for the website. For more examples and to see 
+all available options run:
+
+```
+./site-generator --help
+```

README.md

@@ -4,10 +4,15 @@ Static web page generator for the Hacker Public Radio website.
 ## Installation
 * Clone or download this repository
 * With SQLite
-	* Create the sqlite3 database from the files in the _sql directory. The default name for the database file is "hpr.db" and should be located in the root of the project directory. The name and location can be set in the site.cfg file.
-	* Two sql helper scripts are available to generate an empty database or a database filled with test data.
-		- For an empty database: `cat Create_Database_Empty.sql | sqlite3 hpr.db`
-		- For a database with test data: `cat Create_Database_Test.sql | sqlite3 hpr.db`
+	* Create the sqlite3 database from the hpr.sql MySQL dump file available on
+	  hackerpublicradio.org. The default name for the database file is "hpr.db"
+	  and should be located in the root of the project directory. The name and
+	  location can be set in the site.cfg file.
+	* An "update-hpr.sh" helper script is available in the utils directory. This
+	  script will download the hpr.sql file, convert it to the SQLite hpr.db file,
+	  and regenerate the website using the site-generator.
+		1. `cd` into the root of the project directory
+		2.  Run `./utils/update-hpr.sh`
 	* SQLite v3.8.3 or greater is recommended. CTE WITH clauses are used in some template queries. Must convert WITH
 	  clauses to sub-queries when using earlier versions of SQLite.
 * With MySQL
@@ -26,10 +31,17 @@ Static web page generator for the Hacker Public Radio website.
     * Template
     * Template::Plugin::File
     * Template::Plugin::DBI
+    * Template::Plugin::Date
+    * Template::Plugin::HTML::Strip
     * DBI
     * Tie::DBI
     * DBD::SQLite or DBD:mysql
     * Date::Calc
+    * Text::CSV_XS
+    * HTML::Entities
+
+* See the [Getting Started](GETTING_STARTED.md) tutorial for more details on
+  installing the HPR generator.
 
 ## Usage
 Generate two specific pages:
@@ -60,3 +72,6 @@ and add the label "**Feature Request**".
 
 ## Authors and acknowledgment
 * Roan "Rho`n" Horning
+* gordons
+* Ken Fallon
+* norrist

public_html/css/hpr.css

@@ -1,7 +1,7 @@
 @charset "utf-8";
 /* Website design Copyright Ken Fallon - Released into the public domain/
 
-http://creativecommons.org/publicdomain/ 
+https://creativecommons.org/publicdomain/ 
  
 */
 
@@ -198,7 +198,7 @@ h1.showtitle { font-size: 125%;}
 
 p.author { font-size: 85%;}
 p.shownotes { padding: .5em;}
-#maincontent footer p { font-size: 75%; margin-bottom: 1em;}
+#maincontent footer p { /* font-size: 75%; */ margin-bottom: 1em;}
 /* link styling */
 a:link {text-decoration: none;}
 
@@ -248,6 +248,11 @@ pre.comment {
 	line-height: 1.2rem;
 }
 
+h3.title { margin: 1.25rem 0 0.75rem 0 }
+p.meta { margin: 0.25rem 0 }
+p.summary { margin: 0.25rem 0 }
+p.listen-in { margin-top: 0 }
+
 @media only screen and (max-width: 680px) { 
 	#container {width: 95%;}
 	img#hprlogo {float: left; max-width: 85px; padding: .35em;}

site-generator

@@ -1,80 +1,96 @@
 #!/usr/bin/perl
 
+# {{{ POD documentation
+
 =head1 NAME
 
-	site-generator - HPR Site Generator
+        site-generator - HPR Site Generator
 
 =head1 SYNOPSIS
 
-	site-generator [OPTION]... PAGE|PAGE=<comma separated list of ids>...
+        site-generator [OPTION]... PAGE|PAGE=<comma separated list of ids>...
 
-	-a, --all      generate all pages defined in configuration file
-	-l, --list     print list of configured pages
-	-p, --preview  print generated pages to standard out
-	-q, --quiet    suppress progress information while generating pages
-	-v, --verbose  print extended progress information while generating pages
-	--help         print this help message
+        -a, --all           generate all pages defined in configuration file
+        -c, --configuration path to configuration file
+        -l, --list          print list of configured pages
+        -p, --preview       print generated pages to standard out
+        -q, --quiet         suppress progress information while generating pages
+        -v, --verbose       print extended progress information while generating pages
+        --help              print this help message
 
-	Where I<PAGE> is a file name of a web page 
-	or the special I<ALL> (to generate all pages).
+        Where I<PAGE> is a file name of a web page
+        or the special I<ALL> (to generate all pages).
 
-	Examples:
+        Examples:
 
-		Generate two specific pages:
-		site-generator index about
+                Generate two specific pages:
+                site-generator index about
 
-		Generate the whole site:
-		site-generator --all
+                Generate the whole site:
+                site-generator --all
+
+                Generate pages based on the same template:
+                site-generator correspondent=1,3,5..10
+
+                Generate two specific pages with a different configuration:
+                site-generator --configuration=site_sqlite.cfg index about
 
-		Generate pages based on the same template:
-		site-generator correspondent=1,3,5..10
 
 =head1 DESCRIPTION
 
-This is a site generator for the Hacker Public Radio website based upon the Perl Templates Toolkit.
+This is a site generator for the Hacker Public Radio website based upon the
+Perl Template Toolkit.
 
 =head1 INSTALLATION
 
-	With SQLite
-	* Create the sqlite3 database from the files in the _sql directory. The default name for the 
-	  database file is "hpr.db" and should be located in the root of the project directory. The 
-	  name and location can be set in the site.cfg file.
-	* Two sql helper scripts are available to generate an empty database or a database filled with test data.
-		- For an empty database: cat Create_Database_Empty.sql | sqlite3 hpr.db
-		- For a database with test data: cat Create_Database_Test.sql | sqlite3 hpr.db
-	* SQLite v3.8.3 or greater is recommended. CTE WITH clauses are used in some template queries. 
-	  Must convert WITH clauses to sub-queries when using earlier versions of SQLite.
+        With SQLite
+        * Create the sqlite3 database from the hpr.sql MySQL dump file available on
+          hackerpublicradio.org. The default name for the database file is "hpr.db"
+          and should be located in the root of the project directory. The name and
+          location can be set in the site.cfg file.
+        * An "update-hpr.sh" helper script is available in the utils directory. This
+          script will download the hpr.sql file, convert it to the SQLite hpr.db file,
+          and regenerate the website using the site-generator.
+                1. `cd` into the root of the project directory
+                2.  Run `./utils/update-hpr.sh`
+        * SQLite v3.8.3 or greater is recommended. CTE WITH clauses are used in some template queries.
+          Must convert WITH clauses to sub-queries when using earlier versions of SQLite.
 
-	With MySQL
-	* Create database hpr_hpr in the MySQL server from HPR dump file.
-		- sudo mysql --host=localhost < hpr.sql
-	* Create a user that will be used by the site-generator.
-		- Suggested username: hpr-generator
-		- CREATE USER 'hpr-generator'@'localhost' IDENTIFIED BY '<password>';
-	* Limit the user's privileges to EXECUTE and SELECT
-		- GRANT SELECT ON hpr_hpr.* TO 'hpr-generator'@'localhost';
-		- GRANT EXECUTE ON `hpr_hpr`.* TO 'hpr-generator'@'localhost';
+        With MySQL
+        * Create database hpr_hpr in the MySQL server from HPR dump file.
+                - sudo mysql --host=localhost < hpr.sql
+        * Create a user that will be used by the site-generator.
+                - Suggested username: hpr-generator
+                - CREATE USER 'hpr-generator'@'localhost' IDENTIFIED BY '<password>';
+        * Limit the user's privileges to EXECUTE and SELECT
+                - GRANT SELECT ON hpr_hpr.* TO 'hpr-generator'@'localhost';
+                - GRANT EXECUTE ON `hpr_hpr`.* TO 'hpr-generator'@'localhost';
 
-	Install the needed Perl modules using preferred method (distribution packages, CPAN, etc.)
-		* GetOpt
-		* Pod::Usage
-		* Config::Std
-		* Template
-		* Template::Plugin::File
-		* Template::Plugin::DBI
-		* DBI
-		* Tie::DBI
-		* DBD::SQLite or DBD:mysql
-		* Date::Calc
+        Install the needed Perl modules using preferred method (distribution packages, CPAN, etc.)
+                * Config::Std
+                * DBD::SQLite or DBD:mysql
+                * DBI
+                * Data::Dumper
+                * Date::Calc
+                * GetOpt::Long
+                * HTML::Entities
+                * Pod::Usage
+                * Template
+                * Template::Plugin::DBI
+                * Template::Plugin::Date
+                * Template::Plugin::File
+                * Template::Plugin::HTML::Strip
+                * Text::CSV_XS
+                * Tie::DBI
 
 =head1 AUTHOR
 
-	Roan Horning <roan.horning@no-spam.gmail.com>
+        Roan Horning <roan.horning@no-spam.gmail.com>
 
 =head1 LICENSE
 
-	site-generator -- a static website generator for HPR
-	Copyright (C) 2022 Roan Horning
+        site-generator -- a static website generator for HPR
+        Copyright (C) 2022 Roan Horning
 
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU Affero General Public License as published by
@@ -91,232 +107,332 @@ This is a site generator for the Hacker Public Radio website based upon the Perl
 
 =cut
 
+# }}}
+
+use 5.012;
 use strict;
 use warnings;
+use open ':encoding(UTF-8)';
 
 use Getopt::Long qw(:config auto_help);
 use Pod::Usage;
 use Config::Std;
+use Text::CSV_XS;
+#use HTML::Entities qw(encode_entities_numeric);
+use HTML::Entities qw(encode_entities);
+use Date::Calc;
+use DBI;
+use DBD::SQLite;
+use Tie::DBI;
 use Template;
+use Template::Plugin::Date;
+use Template::Plugin::DBI;
+use Template::Plugin::HTML::Strip;
+
 use Data::Dumper;
 
 exit main();
 
 sub main {
 
-	# Argument parsing
-	my $all;
-	my $preview;
-	my $verbose;
-	my $quiet;
-	GetOptions(
-		'all'     => \$all,
-		'list'    => \&print_available_pages,
-		'preview' => \$preview,
-		'verbose' => \$verbose,
-		'quiet'   => \$quiet,
-	) or pod2usage(1);
-	pod2usage(1) unless @ARGV || $all;
-	my (@page_args) = @ARGV;
+    # Argument parsing
+    my $all;
+    my $configuration_path;
+    my $preview;
+    my $verbose;
+    my $quiet;
+    GetOptions(
+        'all'             => \$all,
+        'configuration=s' => \$configuration_path,
+        'list'            => \&print_available_pages,
+        'preview'         => \$preview,
+        'verbose'         => \$verbose,
+        'quiet'           => \$quiet,
+    ) or pod2usage(1);
+    pod2usage(1) unless @ARGV || $all;
+    my (@page_args) = @ARGV;
 
-	if ($quiet) {
-		$verbose = 'quiet';
-	};
+    if ($quiet) {
+        $verbose = 'quiet';
+    };
 
-	# Load config file
-	read_config "site.cfg" => my %config;
+    if (!$configuration_path) {
+        $configuration_path = "site.cfg";
+    }
 
-	my $tt = get_template_html($config{DBI});
+    my %config;
+    if ( -f $configuration_path ) {
+        # Load config file
+        read_config $configuration_path => %config;
+    }
+    else {
+        print STDOUT "Could not read configuration file: $configuration_path\n";
+        exit 1;
+    }
 
-	# If command line option all is set, parse configuration file 
-	# for all pages
-	if ($all) {
-		@page_args = keys %config;
+    my $tt = get_template_html($config{DBI}, $config{app_paths});
 
-		# Remove non page sections of the configuration file
-		# from the generated list of pages.
-		@page_args= grep { $_ ne 'DBI' } @page_args;
-		@page_args= grep { $_ ne 'root_template' } @page_args;
-		@page_args= grep { $_ ne 'media_hostname' } @page_args;
+    #
+    # Define a TT² vmethod called 'csv_parse', it takes a scalar value and
+    # returns an arrayref. Also define a filter called 'xml_entity' which
+    # numerically encodes non-ASCII characters.
+    #
+    $tt->context->define_vmethod( 'scalar', 'csv_parse', \&parse_csv );
+    $tt->context->define_filter( 'xml_entity', \&xml_entity );
 
-	};
-	foreach my $page_arg (@page_args) {
-		my %parsed_arg = parse_page_arg($page_arg);
-		if (exists($config{$parsed_arg{'page'}})) {
-			my $page_config = $config{$parsed_arg{'page'}}; 
-			$page_config->{'page'} = $parsed_arg{'page'};
+    # If command line option all is set, parse configuration file
+    # for all pages
+    if ($all) {
+        @page_args = keys %config;
 
-			# Set page's root_template to the default root_template if the 
-			# page root_template property is not set in the configuration file.
-			if (exists $page_config->{'root_template'} == 0) {
-				$page_config->{'root_template'} = $config{root_template}{content};
-			}
+        # Remove non page sections of the configuration file
+        # from the generated list of pages.
+        @page_args= grep { $_ ne 'DBI' } @page_args;
+        @page_args= grep { $_ ne 'root_template' } @page_args;
+        @page_args= grep { $_ ne 'app_paths' } @page_args;
 
-			# Set all config root_template properties as default page config properties
-			# except the previously set root_template content property
-			my @root_args = grep { $_ ne 'content' } keys %{$config{root_template}};
-			foreach my $root_arg (@root_args) {
-				if (exists $page_config->{$root_arg} == 0) {
-					$page_config->{$root_arg} = $config{root_template}{$root_arg};
-				}
-			}
+    };
+    foreach my $page_arg (@page_args) {
+        my %parsed_arg = parse_page_arg($page_arg);
+        if (exists($config{$parsed_arg{'page'}})) {
+            my $page_config = $config{$parsed_arg{'page'}};
+            $page_config->{'page'} = $parsed_arg{'page'};
 
-			if ($page_config->{'multipage'} && $page_config->{'multipage'} eq 'true') {
-				if (scalar @{$parsed_arg{'ids'}} == 1) {
-				  @{$parsed_arg{'ids'}} = get_ids_from_db($tt, \$page_config);	
-				}
-				foreach my $id (@{$parsed_arg{'ids'}}) {
-					$page_config->{'id'} = $id;
-					verbose ($verbose, "Generating page: $page_config->{'page'} with id: $id");
-					generate_page($tt, \$page_config, $preview);
-				}
-			}
-			else {
-				verbose ($verbose, "Generating page: $page_config->{'page'}");
-				generate_page($tt, \$page_config, $preview);
-			}
-		}
-		else {
-			verbose (1, "\nWarning: Page $parsed_arg{'page'} is not defined in the configuration file.");
-		}
-	}
+            # Set page's root_template to the default root_template if the
+            # page root_template property is not set in the configuration file.
+            if (exists $page_config->{'root_template'} == 0) {
+                $page_config->{'root_template'} = $config{root_template}{content};
+            }
+
+            # Set all config root_template properties as default page config properties
+            # except the previously set root_template content property
+            my @root_args = grep { $_ ne 'content' } keys %{$config{root_template}};
+            foreach my $root_arg (@root_args) {
+                if (exists $page_config->{$root_arg} == 0) {
+                    $page_config->{$root_arg} = $config{root_template}{$root_arg};
+                }
+            }
+
+            if ($page_config->{'multipage'} && $page_config->{'multipage'} eq 'true') {
+                if (scalar @{$parsed_arg{'ids'}} == 0) {
+                    @{$parsed_arg{'ids'}} = get_ids_from_db($tt, \$page_config);
+                }
+                foreach my $id (@{$parsed_arg{'ids'}}) {
+                    $page_config->{'id'} = $id;
+                    verbose ($verbose, "Generating page: $page_config->{'page'} with id: $id");
+                    generate_page($tt, \$page_config, $preview);
+                }
+            }
+            else {
+                verbose ($verbose, "Generating page: $page_config->{'page'}");
+                generate_page($tt, \$page_config, $preview);
+            }
+        }
+        else {
+            verbose (1, "\nWarning: Page $parsed_arg{'page'} is not defined in the configuration file.");
+        }
+    }
     verbose (1, "\nFinished processing the files.");
-	return 0;
+    return 0;
 }
 
-sub get_template_html (\%@)  {
-	# For an HTML based Template file, define the 
-	# template start and end tags to also function as 
-	# HTML comments to make the template file valid HTML.
-	#
-	return Template->new({
-			INCLUDE_PATH => './templates',
-			OUTPUT_PATH	 => './public_html',
-			EVAL_PERL    => 1,
-			START_TAG	 => '<!--%',
-			END_TAG		 => '%-->',
-			PRE_CHOMP  => 1,
-			POST_CHOMP => 1,
-			CONSTANTS    => {
-				database => $_[0]{database},
-				driver   => $_[0]{driver},
-				user     => $_[0]{user},
-				password => $_[0]{password},
-			}
-		}) || die $Template::ERROR, "\n";
+sub get_template_html {
+    # For an HTML based Template file, define the
+    # template start and end tags to also function as
+    # HTML comments to make the template file valid HTML.
+    #
+    return Template->new(
+        {   INCLUDE_PATH => $_[1]{templates_path},
+            OUTPUT_PATH  => $_[1]{output_path},
+            EVAL_PERL    => 1,
+            START_TAG    => '<!--%',
+            END_TAG      => '%-->',
+            PRE_CHOMP    => 1,
+            POST_CHOMP   => 1,
+            CONSTANTS    => {
+                database => $_[0]{database},
+                driver   => $_[0]{driver},
+                user     => $_[0]{user},
+                password => $_[0]{password},
+            }
+        }
+    ) || die $Template::ERROR, "\n";
 
 }
 
-