From add4f338be0afc738f95ed51738cb98525d5f6c9 Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Sat, 4 Mar 2023 09:10:06 -0500 Subject: [PATCH 01/11] Add Getting Started tutorial Currently a very high level outline of the document. --- GETTING_STARTED.md | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 GETTING_STARTED.md diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md new file mode 100644 index 0000000..1c39252 --- /dev/null +++ b/GETTING_STARTED.md @@ -0,0 +1,9 @@ +# Clone the repository + +# Install required Perl modules + +# Create HPR database + +# Configure site-generator + +# Run site-generator \ No newline at end of file From 16d6cf99aa11656c0f2f9607c9d0d87e6c5f319d Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Sat, 4 Mar 2023 09:27:56 -0500 Subject: [PATCH 02/11] Add instructions for cloning the repository --- GETTING_STARTED.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 1c39252..0eedf6a 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -1,5 +1,16 @@ # Clone the repository +If you do not have git installed on your system. Please install it now +(see the git documentation for [instructions on installing git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)). + +To retrieve the code from the repository on anhonesthost.net run: + +``` +clone https://repo.anhonesthost.net/rho_n/hpr_generator.git +``` +On success, an "hpr_generator" directory will be created in the folder from +which the clone command was executed containing a local copy of the git repository. + # Install required Perl modules # Create HPR database From 82d761bf68d40de56b6a85db84e1b216e8ea53fb Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Tue, 7 Mar 2023 18:53:03 -0500 Subject: [PATCH 03/11] Update Clone the Repository section Add reference to the gitlab.com mirror repository. --- GETTING_STARTED.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 0eedf6a..5a7583a 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -1,13 +1,16 @@ # Clone the repository -If you do not have git installed on your system. Please install it now +If git is not installed on the operating system, please install it now (see the git documentation for [instructions on installing git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)). -To retrieve the code from the repository on anhonesthost.net run: +To retrieve the code from the repository on anhonesthost.net, run: + +`clone https://repo.anhonesthost.net/rho_n/hpr_generator.git` + +To retrieve the code from the repository on gitlab.com, run: + +`https://gitlab.com/roan.horning/hpr_generator.git` -``` -clone https://repo.anhonesthost.net/rho_n/hpr_generator.git -``` On success, an "hpr_generator" directory will be created in the folder from which the clone command was executed containing a local copy of the git repository. From d6068778ab04419550d59120dfd9d8838e36d253 Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Tue, 7 Mar 2023 18:55:08 -0500 Subject: [PATCH 04/11] Add instructions for installing Perl modules --- GETTING_STARTED.md | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 5a7583a..7ebcd1c 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -16,7 +16,41 @@ which the clone command was executed containing a local copy of the git reposito # Install required Perl modules -# Create HPR database +Installing the Perl modules is the most finicky part of the installation process. +The needed Perl modules can be found using the operating systems's package +manager or using the modules found in the +[Comprehensive Perl Archive Network (CPAN)](https://www.cpan.org/). + +## Installing modules on Debian based Linux distributions + +Run command (tested on Debian 11): + +``` +apt install libconfig-std-perl \ + libtemplate-perl \ + libtemplate-plugin-dbi-perl \ + libdbd-sqlite3-perl libdate-calc-perl \ + libtie-dbi-perl +``` + +## Using CPAN to install the modules + +A cross platform method to install the needed modules is Perl CPAN application. +Make sure both the [make](https://www.gnu.org/software/make/manual/make.html) +command and the [cpan](https://perldoc.perl.org/CPAN) command are available. +Install them using the operating system's package manager, or from source. + +Run commands: + +``` +cpan Config::Std +cpan Template +cpan Template::Plugin::DBI +cpan DBD::SQLite +cpan Date::Calc +cpan Tie::DBI +``` + # Configure site-generator From 7feae31e11600f3cf7c098f3419723ea4090dc3e Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Tue, 7 Mar 2023 19:03:08 -0500 Subject: [PATCH 05/11] Add helper script that creates the HPR SQLite file --- utils/update-hpr-db.sh | 54 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100755 utils/update-hpr-db.sh diff --git a/utils/update-hpr-db.sh b/utils/update-hpr-db.sh new file mode 100755 index 0000000..765c9fc --- /dev/null +++ b/utils/update-hpr-db.sh @@ -0,0 +1,54 @@ +#!/bin/bash - +#=============================================================================== +# +# FILE: update-hpr-db.sh +# +# USAGE: ./update-hpr-db.sh +# +# DESCRIPTION: Download MySQL hpr.sql file and create SQLite3 file +# +# OPTIONS: --- +# REQUIREMENTS: lib_utils.sh +# BUGS: --- +# NOTES: --- +# AUTHOR: Roan "Rho`n" Horning (roan.horning@gmail.com) +# ORGANIZATION: +# CREATED: 03/05/2023 07:21:29 PM +# REVISION: --- +# LICENSE: GNU AGPLv3 +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . +# +#=============================================================================== + +set -o nounset # Treat unset variables as an error + +BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# +# Load library functions +# +LIB="$BASEDIR/lib_utils.sh" +[ -e $LIB ] || { echo "Unable to load functions.\n$LIB not found."; exit; } +source $LIB + +WORKING_DIR=`make_working_dir` + +download_hpr_sql $WORKING_DIR + +make_hpr_sqlite_db $WORKING_DIR + +copy_to_public_dir $WORKING_DIR `pwd` + +clean_working_dir $WORKING_DIR From b420453abd47c7f1b3a5fbf827591c7c90cbb47c Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Tue, 7 Mar 2023 19:06:29 -0500 Subject: [PATCH 06/11] Add instructions for creating HPR SQLite file --- GETTING_STARTED.md | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 7ebcd1c..d223196 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -51,7 +51,41 @@ cpan Date::Calc cpan Tie::DBI ``` +# Create the HPR database + +The hpr_generator relies on information from a database to generate many of the +files for the website (for example: index.html, series/index.html, +hpr_mp3.rss, etc). This data is available from a MySQL dump file found on +hackerpublicradio.org at "https://www.hackerpublicradio.org/hpr.sql". + +The first step is to download the hpr.sql file. This can be done using your +browser, or by running one of the following commands: + +`curl https://www.hackerpublicradio.org/hpr.sql --output ./hpr.sql` + +or + +`wget --directory-prefix=./ https://www.hackerpublicradio.org/hpr.sql` + +## Creating an SQLite database file + +The SQL of the hpr.sql file must be converted from MySQL specific statements to +SQLite specific statements. The mysql2sqlite script found in the utils directory +is used for this conversion. First remove the lines from hpr.sql that +mysql2sqlite can't handle: + +`sed '/^DELIMITER ;;/,/^DELIMITER ;/d' < ./hpr.sql > ./hpr-sqlite.sql` + +Next run the mysql2sqlite script piping its output into the sqlite3 +program which creates the hpr.db file: + +`./utils/mysql2sqlite ./hpr-sqlite.sql | sqlite3 ./hpr.db` + +For convenience, the update-sqlite-db.sh script in the utils directory +automates the above steps (including downloading the hpr.sql file). +From the root of the local hpr_generator repository run: + +`./utils/update-sqlite-db.sh` -# Configure site-generator # Run site-generator \ No newline at end of file From 97a5d4ebbda0f89f79bdbc33cbbd1fc7ac25fe08 Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Tue, 7 Mar 2023 19:09:33 -0500 Subject: [PATCH 07/11] Add instructions for configuring the site-generator --- GETTING_STARTED.md | 98 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 1 deletion(-) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index d223196..998e829 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -87,5 +87,101 @@ From the root of the local hpr_generator repository run: `./utils/update-sqlite-db.sh` +# Configure the site-generator + +In your favorite text editor, open the site.cfg file found in the root of the +"hpr_generator" folder. Full details about options for configuring the site.cfg +file are found in the comments within the file. + +## Configuring the database connection + +Any database supported by the Perl:DBI and Perl::DBD modules can be used with +the site-generator program. Currently the hpr_generator project works with +a MySQL or SQLite database. + +Find the [DBI] section of the file. It should look like the following + +``` +[DBI] +# Configuration settings for SQLite +#database: sqlite +#driver: dbi:SQLite:hpr.db +#user: (not used - leave blank) +#password: (not used - leave blank) +# Configuration settings for MySQL +#database: mysql +#driver: dbi:mysql:database=hpr_hpr:hostname=localhost +#user: hpr-generator (Suggested user with read-only privileges) +#password: ********* (Password for user) +``` + +### SQLite + +Remove the comment character from the start of the database and driver +option lines: + + ``` +# Configuration settings for SQLite +database: sqlite +driver: dbi:SQLite:hpr.db +#user: (not used - leave blank) +#password: (not used - leave blank) +``` + +The hpr.db section of the driver option `dbi:SQLite:hpr.db` is the path +to the sqlite file. The default assumes the hpr.db file is located in the same +directory as the site-generator. + +### MySQL + +Remove the comment character from the start of the database, driver, +user, and password option lines: + + ``` +# Configuration settings for MySQL +database: mysql +driver: dbi:mysql:database=hpr_hpr:hostname=localhost +user: hpr-generator +password: ********* +``` + +This assumes that the MySQL database service is available at the localhost +hostname, that the database name (hpr_hpr) is the database created from +the hpr.sql dump file or manually created by you, that the user (hpr-generator) +was added by you and has read rights to the hpr_hpr database, and that the +password (replace ********* with the actual password) matches the password set +for the hpr-generator database user. + +## Configuring the website for viewing locally + +For HTML links to work when viewing the files on your local machine using the +"file://" protocal (i.e. using the "Open..." command in your browser, each HTML +file must include a \ meta-data tag in the \ section of its +contents. To configure this in the site.cfg file, find the [root_template] +section. It should look like the following: + +``` +[root_template] +content: page.tpl.html +#baseurl: OPTIONAL [i.e. file://] +``` +Below the #baseurl comment line add: + +``` +baseurl: file:///hpr_generator/public_html +``` + +Replace \ with the full path to the hpr_generator directory. For +example: `file:///home/rho_n/development/hpr_generator/public_html` + +## Configuring the website media file links + +If you do not want to host all the media files (currently, audio files and +transcription files), you can configure the `media_baseurl` option. This can +be added to the [root_template] section of the site.cfg file. Suggested +external site is archive.org. To use this site add: + +``` +media_baseurl: https://archive.org/download/hpr$eps_id/ +``` -# Run site-generator \ No newline at end of file From 9310a8642d17fee4232c548e2249004a6edc492d Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Tue, 7 Mar 2023 19:11:19 -0500 Subject: [PATCH 08/11] Add instructions for running the site-configurator --- GETTING_STARTED.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 998e829..866340d 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -185,3 +185,17 @@ external site is archive.org. To use this site add: media_baseurl: https://archive.org/download/hpr$eps_id/ ``` +# Run the site-generator + +Run the site generator form the hpr_generator directory: + +``` +./site-generator --all +``` + +This will generate all the files for the website. For more examples and to see +all available options run: + +``` +./site-generator --help +``` From 231a50957f4318988f6c94116cb361fa2704a798 Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Tue, 7 Mar 2023 19:33:09 -0500 Subject: [PATCH 09/11] Fix clone command of repositories --- GETTING_STARTED.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 866340d..6603593 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -5,11 +5,11 @@ If git is not installed on the operating system, please install it now To retrieve the code from the repository on anhonesthost.net, run: -`clone https://repo.anhonesthost.net/rho_n/hpr_generator.git` +`git clone https://repo.anhonesthost.net/rho_n/hpr_generator.git` To retrieve the code from the repository on gitlab.com, run: -`https://gitlab.com/roan.horning/hpr_generator.git` +`git clone https://gitlab.com/roan.horning/hpr_generator.git` On success, an "hpr_generator" directory will be created in the folder from which the clone command was executed containing a local copy of the git repository. From 207eabec86f465b450bdc92e7c479a80219ec0b1 Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Thu, 9 Mar 2023 21:39:53 -0500 Subject: [PATCH 10/11] Add reference to the file GETTING_STARTED.md Update the Installation instructions to include reference to the Getting Started tutorial. --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index fff337a..55fee63 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,8 @@ Static web page generator for the Hacker Public Radio website. * Tie::DBI * DBD::SQLite or DBD:mysql * Date::Calc - +* See the Getting Started tutorial (GETTING_STARTED.md) for more details on + installing the HPR generator. ## Usage Generate two specific pages: `site-generator index about` From fc7975379ed920687d0f2a9d60cd3c389a45950d Mon Sep 17 00:00:00 2001 From: Roan Horning Date: Thu, 9 Mar 2023 21:42:36 -0500 Subject: [PATCH 11/11] Fix grammer issues --- GETTING_STARTED.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 6603593..5d7da38 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -3,7 +3,8 @@ If git is not installed on the operating system, please install it now (see the git documentation for [instructions on installing git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)). -To retrieve the code from the repository on anhonesthost.net, run: +To retrieve the code from the repository on anhonesthost.net (a username and +password are required), run: `git clone https://repo.anhonesthost.net/rho_n/hpr_generator.git` @@ -17,7 +18,7 @@ which the clone command was executed containing a local copy of the git reposito # Install required Perl modules Installing the Perl modules is the most finicky part of the installation process. -The needed Perl modules can be found using the operating systems's package +The needed Perl modules can be found using the operating system's package manager or using the modules found in the [Comprehensive Perl Archive Network (CPAN)](https://www.cpan.org/). @@ -35,7 +36,7 @@ apt install libconfig-std-perl \ ## Using CPAN to install the modules -A cross platform method to install the needed modules is Perl CPAN application. +A cross platform method to install the needed modules is the Perl CPAN application. Make sure both the [make](https://www.gnu.org/software/make/manual/make.html) command and the [cpan](https://perldoc.perl.org/CPAN) command are available. Install them using the operating system's package manager, or from source. @@ -56,7 +57,7 @@ cpan Tie::DBI The hpr_generator relies on information from a database to generate many of the files for the website (for example: index.html, series/index.html, hpr_mp3.rss, etc). This data is available from a MySQL dump file found on -hackerpublicradio.org at "https://www.hackerpublicradio.org/hpr.sql". +hackerpublicradio.org at URL "https://www.hackerpublicradio.org/hpr.sql". The first step is to download the hpr.sql file. This can be done using your browser, or by running one of the following commands: