InternetArchive/recover_transcripts: Bash script to be run on 'borg'
which collects files missing on the IA ready for upload as part of
the missing asset repair process.
InternetArchive/repair_assets: Bash script to take assets from the IA
(after they had been repaired on 'borg') and copy them to the HPR
server for the notes to access. The local machine, where this was
run, was used to store files being uploaded. The planned script to
modify the notes to reflect the new file locations was never
finished. Notes were edited with Vim using a few macros.
InternetArchive/repair_item: Bash script which is best run on 'borg',
which repairs an IA item by comparing the files on the IA with the
files on 'borg' (or a local machine). These files are either in
'/data/IA/uploads/' or in the temporary file hierarchy used by
'recover_transcripts' (which calls it). Used after a normal IA
upload to check for and make good any missed file uploads (due to
timeouts, etc). Also used during asset repairs, but that project is
now finished.
InternetArchive/snapshot_metadata: Bash script which collects detailed
metadata from the IA in JSON format and saves it locally (run on
a local PC). Older shows on the IA often contained derivative files
which were identified by the script 'view_derivatives'. These files
were never needed, they were IA artefacts, so can be deleted (see
the script header for how).
InternetArchive/view_derivatives: Perl script to interpret a file of
JSON metadata from the IA for an HPR show in order to determine the
parent-child hierarchy of files where there may be derivatives. We
don't want IA-generated derivatives, but this process was hard to
turn off in earlier times. Generates a hierarchical report and
a list of unwanted derivatives (see 'snapshot_metadata' for more
details of how this was used).
InternetArchive/repair_assets: Accidentally reverset the "sanity check"
logic, so put it back the right way!
InternetArchive/view_derivatives: Started on the POD documentation but
didn't get very far.
InternetArchive/view_derivatives: New Perl script which Reads JSON
metadata from the IA and builds tree-like structures linking
original and derived files on the IA. It reports these trees and
saves a subset of derived files in an output file to be used for
deletion. In general we do not want derivatives, we make them
ourselves. Older software had no reliable way to prevent them.
InternetArchive/future_upload: Added logging and debugging
InternetArchive/ia_db.sql: Added new tables
InternetArchive/recover_transcripts: New script to run on 'borg' and
copy missing files from the backup disk to the IA
InternetArchive/repair_assets: More comments, including one about a bug in the design.
InternetArchive/repair_item: Fix relating to octal numbers (if there are
leading zeroes in a number). '_DEBUG' is now in the function
library. Added comments to explain obscure stuff.
InternetArchive/snapshot_metadata: New Bash script (to run on my
desktop) which collects metadata for a show and stores in in the
'~/HPR/IA/assets' directory. Runs 'view_derivatives' on it to find
derivative files for deletion.
InternetArchive/tidy_uploaded: Moves files and directories containing
uploaded files into a holding area for later backup. Added
debugging, logging and a 'force' mode.
InternetArchive/upload_manager: Manages 'ia.db' (on my workstation).
Needs many updates which have just started to be added.
InternetArchive/weekly_upload: Old script, now obsolete.
Database/query2tt2: comment and documentation updates; use of Perl's
try/catch.
InternetArchive/.make_metadata.cfg: added comments for readability
InternetArchive/make_metadata: bug fix needed now that all shows on the HPR server have
a directory with assets under it.
InternetArchive/repair_assets: new Bash script in development. Collects
assets from the IA and uploads them to a new directory on the HPR
server. Will run 'fix_asset_links' (to repair asset links for their
new directories) once it is ready.
InternetArchive/repair_item: Bash script which was originally written to
run on 'borg' and upload files to a new IA item when the uploads
timed out. Now enhanced to upload missing files recovered from the
HPR backup disk, such as transcripts.
InternetArchive/function_lib.sh: new file; subset of
'~/bin/function_lib.sh' which is referred to in a number of scripts.
It contains relevant functions such as 'yes_no' and 'define_colours'.
InternetArchive/repair_item: originally planned in 2020 as a Bash script
to find missing files in shows and then add them, it was not turned
into the current form until May 2024. Now, with the heavy loading of
the IA servers, normal uploads are timing out and being aborted.
This script is more "determined" to upload files and usually
successfully "repairs" shows that need it.
FAQ/FAQ.mkd, FAQ/Makefile: this version of the FAQ is now out of date
and probably should be deleted.
InternetArchive/repair_item: script to upload missing shows after tie
out errors during the normal upload; still under development.
InternetArchive/update_state: script to update show state in the
'reservations' table in the database. Uses the CMS interface.
Link_Checker/scan_links: under development. Not currently usable.
Miscellaneous/fix_tags: audio metadata manipulation script. Recently
added to this repo for convenience. Updates for 'experimental::try',
the official Perl try/catch.
PostgreSQL_Database/add_hosts_to_show, PostgreSQL_Database/hpr_schema_2.pgsql,
PostgreSQL_Database/nuke_n_pave.sh: an old experimental Pg database
to take over from the previous MySQL version (from before 2023).
Kept for reference; never implemented.
