sgoti/hpr-tools
1
0
Fork 0
forked from HPR/hpr-tools
Code Pull Requests Activity

Updates for show "repair" processing

Browse Source 
InternetArchive/future_upload: Added logging and debugging

InternetArchive/ia_db.sql: Added new tables

InternetArchive/recover_transcripts: New script to run on 'borg' and
    copy missing files from the backup disk to the IA

InternetArchive/repair_assets: More comments, including one about a bug in the design.

InternetArchive/repair_item: Fix relating to octal numbers (if there are
    leading zeroes in a number). '_DEBUG' is now in the function
    library. Added comments to explain obscure stuff.

InternetArchive/snapshot_metadata: New Bash script (to run on my
    desktop) which collects metadata for a show and stores in in the
    '~/HPR/IA/assets' directory. Runs 'view_derivatives' on it to find
    derivative files for deletion.

InternetArchive/tidy_uploaded: Moves files and directories containing
    uploaded files into a holding area for later backup. Added
    debugging, logging and a 'force' mode.

InternetArchive/upload_manager: Manages 'ia.db' (on my workstation).
    Needs many updates which have just started to be added.

InternetArchive/weekly_upload: Old script, now obsolete.
This commit is contained in:
Dave Morriss
2024-08-22 13:13:38 +01:00
parent dc0f29e957
commit 19030fee71
9 changed files with 994 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
InternetArchive/repair_assets
View File

@@ -15,15 +15,15 @@
# BUGS: ---
# NOTES: ---
# AUTHOR: Dave Morriss (djm), Dave.Morriss@gmail.com
# VERSION: 0.0.6
# VERSION: 0.0.7
# CREATED: 2024-05-10 21:26:31
# REVISION: 2024-07-10 15:12:54
# REVISION: 2024-08-04 19:40:52
#
#===============================================================================
# set -o nounset # Treat unset variables as an error
VERSION="0.0.6"
VERSION="0.0.7"
SCRIPT=${0##*/}
# DIR=${0%/*}
@@ -357,7 +357,7 @@ else
exit 1
fi
_DEBUG "$(printf '%s\n' "${iacache[@]}")"
_DEBUG "IA cache" "${iacache[@]}"
#
# Determine which files are assets
@@ -461,6 +461,10 @@ fi
#-------------------------------------------------------------------------------
# Compare the two asset lists and return what's missing on the HPR server
#-------------------------------------------------------------------------------
# TODO: This algorithm does not handle the instance where there are pictures
# in one directory and a lower directory containing thumbnails, AND THE FILE
# NAMES ARE THE SAME!
#
declare -a missing
find_missing ia_asset hpr_asset missing
_verbose "$(coloured 'cyan' "** missing (${#missing[@]}):")"
@@ -471,7 +475,8 @@ if [[ ${#missing[@]} -eq 0 ]]; then
_log "No missing assets detected; nothing to do"
exit
else
coloured 'yellow' "Found ${#missing[@]} files missing on the HPR server"
coloured 'yellow' \
"Found ${#missing[@]} $(ngettext file files ${#missing[@]}) missing on the HPR server"
fi
#-------------------------------------------------------------------------------
@@ -548,6 +553,10 @@ fi
#-------------------------------------------------------------------------------
# Synchronise assets to the directory
#-------------------------------------------------------------------------------
# We perform an 'rsync' over 'ssh' to synchronise files from
# ~/HPR/InternetArchive/assets/hprXXXX/files/hprXXXX to
# public_html/eps/hprXXXX (on the HPR server)
#
# shellcheck disable=SC2059 disable=SC2089
printf -v command "$RSYNCTPL" "$LOCAL_PARENTDIR/" "$REMOTE_PARENTDIR/"