HPR/hpr-tools
8
1
Fork 2
Code Issues 14 Pull Requests Packages Projects Releases Wiki Activity

Occasional download failure of database snapshot 'hpr.sql' #17

New Issue
Open
opened 2025-11-02 18:00:37 +00:00 by davmo · 0 comments
davmo commented 2025-11-02 18:00:37 +00:00
Owner
Copy Link

When attempting to download this file it is possible to receive the following error:

--2025-11-01 19:07:36--  https://www.hackerpublicradio.org/hpr.sql
Resolving www.hackerpublicradio.org (www.hackerpublicradio.org)... 204.13.239.180
Connecting to www.hackerpublicradio.org (www.hackerpublicradio.org)|204.13.239.180|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2025-11-01 19:07:36 ERROR 403: Forbidden.

It seems likely that the file is unavailable because it is being written at that moment.

It would be better if the process that generates the file used the more resilient approach which makes the update "atomic". In general this consists of:

  • Writing new data to a temporary file
  • Changing its permissions in line with the downloadable file
  • Renaming it using mv to overwrite the existing file with the new one, this is an atomic process

See the StackExchange discussion of this: rewrite existing file so that it gets replaced by new version atomically, only once fully written

When attempting to download this file it is possible to receive the following error: ``` --2025-11-01 19:07:36-- https://www.hackerpublicradio.org/hpr.sql Resolving www.hackerpublicradio.org (www.hackerpublicradio.org)... 204.13.239.180 Connecting to www.hackerpublicradio.org (www.hackerpublicradio.org)|204.13.239.180|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2025-11-01 19:07:36 ERROR 403: Forbidden. ``` It seems likely that the file is unavailable because it is being written at that moment. It would be better if the process that generates the file used the more resilient approach which makes the update "atomic". In general this consists of: - Writing new data to a temporary file - Changing its permissions in line with the downloadable file - Renaming it using `mv` to overwrite the existing file with the new one, this is an _atomic_ process See the StackExchange discussion of this: [rewrite existing file so that it gets replaced by new version atomically, only once fully written](https://unix.stackexchange.com/questions/24395/rewrite-existing-file-so-that-it-gets-replaced-by-new-version-atomically-only-o)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
archer72 davmo jknapp ken_fallon norrist paulj rho_n sgoti
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: HPR/hpr-tools#17
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?