HPR/hpr-tools
8
1
Fork 2
Code Issues 10 Pull Requests Packages Projects Releases Wiki Activity

Occasional download failure of database snapshot 'hpr.sql' #17

New Issue
Closed
opened 2025-11-02 18:00:37 +00:00 by davmo · 1 comment
davmo commented 2025-11-02 18:00:37 +00:00
Owner
Copy Link
Copy Source

When attempting to download this file it is possible to receive the following error:

--2025-11-01 19:07:36--  https://www.hackerpublicradio.org/hpr.sql
Resolving www.hackerpublicradio.org (www.hackerpublicradio.org)... 204.13.239.180
Connecting to www.hackerpublicradio.org (www.hackerpublicradio.org)|204.13.239.180|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2025-11-01 19:07:36 ERROR 403: Forbidden.

It seems likely that the file is unavailable because it is being written at that moment.

It would be better if the process that generates the file used the more resilient approach which makes the update "atomic". In general this consists of:

  • Writing new data to a temporary file
  • Changing its permissions in line with the downloadable file
  • Renaming it using mv to overwrite the existing file with the new one, this is an atomic process

See the StackExchange discussion of this: rewrite existing file so that it gets replaced by new version atomically, only once fully written

When attempting to download this file it is possible to receive the following error: ``` --2025-11-01 19:07:36-- https://www.hackerpublicradio.org/hpr.sql Resolving www.hackerpublicradio.org (www.hackerpublicradio.org)... 204.13.239.180 Connecting to www.hackerpublicradio.org (www.hackerpublicradio.org)|204.13.239.180|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2025-11-01 19:07:36 ERROR 403: Forbidden. ``` It seems likely that the file is unavailable because it is being written at that moment. It would be better if the process that generates the file used the more resilient approach which makes the update "atomic". In general this consists of: - Writing new data to a temporary file - Changing its permissions in line with the downloadable file - Renaming it using `mv` to overwrite the existing file with the new one, this is an _atomic_ process See the StackExchange discussion of this: [rewrite existing file so that it gets replaced by new version atomically, only once fully written](https://unix.stackexchange.com/questions/24395/rewrite-existing-file-so-that-it-gets-replaced-by-new-version-atomically-only-o)
ken_fallon commented 2026-06-22 19:29:33 +00:00
Owner
Copy Link
Copy Source

This has been re-engineered

This has been re-engineered
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
HPR
Bug found
2
For issues related to the code or the content of files
HPR
Feature Request
Used when requesting a new Feature.
HPR
General discussion
1
For discussing issues or ideas for HPR content or features
HPR
Help Request
4
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
archer72 davmo (Dave Morriss) jknapp ken_fallon (Ken Fallon) norrist paulj (Paul Jewell ) rho_n (Roan Horning) sgoti
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: HPR/hpr-tools#17