2024-10-29_11-49-56_CET
parent
95066680dc
commit
b863ddf24a
13
Processing-Show-Notes.md
Normal file
13
Processing-Show-Notes.md
Normal file
@ -0,0 +1,13 @@
|
||||
- A `cron` job runs periodically to flag whether there are any notes requiring work (`cronjob_scrape`). The HPR website is scraped with a Perl script to determine this (`scrape_HPR`) by looking for entries in the 'processing' state on the calendar page.
|
||||
- If there is work to do the partial copy of the `upload` directory on the HPR server kept locally is synchronised with `rsync` over SSH (`sync_hpr`).
|
||||
- Files for new shows are saved locally in a directory called `'hprXXXX'`, based on the show number. A record is kept of the `upload/` sub-directory where the show came from so that the end result can be uploaded to it (in the file `.origin`).
|
||||
- For each new show the following steps are carried out:
|
||||
- The raw `shownotes.txt` file is viewed so it can be checked for errors (`do_show`). This is the point at which errors like misspellings in the title, summary or tags can be determined. These are corrected by editing the raw file and re-uploading it. The script used to perform such edits is `do_repair`.
|
||||
- The `shownotes.txt` file is parsed. The declared note format is saved for future reference in a file called `.format`, and the notes themselves are stored in a file called `'hprXXXX.out'`. Parsing is controlled by `do_parse` which calls a Perl script `parse_shownotes`. Any obvious anomalies such as missing media, summary or tags are flagged at this stage by the Perl script. This script also tries to determine whether the declared format (e.g. HTML5) matches the actual note content, and flags any apparent errors.
|
||||
- It is possible to change the declared format at this stage if it seems appropriate (e.g. it's **not** HTML, just plain text) using script `do_change_format`.
|
||||
- The show notes can then be edited. This is done with `do_vim`. This script passes the declared file format to Vim in order to enable the relevant syntax. If the format is 'HTML5' a validator is run on the notes (script `validate_html`). If there are errors these are passed to Vim so that the problems can be found and corrected. For other formats an external script (`make_markdown`) can be used to convert selected parts or the whole file to Markdown when desired.
|
||||
- Having produced suitable Markdown (as appropriate) or other format, the notes can be converted to HTML5 using `pandoc`. This stage is skipped if the notes are already HTML5. The script used is called `do_pandoc`. Two types of files are generated by this script: the first is the HTML fragment destined for the HPR database, called `'hprXXXX.html'`; the second is for local consumption and is a full standalone file which uses the HPR CSS, called `'hprXXXX_full.html'`. The `do_pandoc` script passes `pandoc` various settings according to the format of the input file and the desired output file.
|
||||
- The HTML is viewed with the script `do_browser` (which is actually a soft link to another script tailored for the particular preferred browser; currently the browser is `brave` and the script is `do_brave`).
|
||||
- There is usually an iteration between editing, running `pandoc` and viewing in the browser before the notes are accepted.
|
||||
- The final stage is to run `do_upload` which copies the HTML fragment to the HPR site under the appropriate `upload/` sub-directory.
|
||||
- The saved show files are deleted by a cron job according to their age. Currently they are stored for 6 months.
|
||||
52
home.md
Normal file
52
home.md
Normal file
@ -0,0 +1,52 @@
|
||||
# HPR Workflow
|
||||
|
||||
The HPR website is moving to a distributed system where the entire site, can be easily replicated by anyone. With the exception of login credentials, it should be possible for anyone to download and run everything needed to host HPR.
|
||||
|
||||
```mermaid
|
||||
stateDiagram
|
||||
[*] --> s1
|
||||
state "Host Requests a slot" as s1
|
||||
state "Slot is reserved for a short time" as s1
|
||||
state "request.php" as s1
|
||||
s1 --> s2
|
||||
state "Email is sent to host" as s2
|
||||
state "request_confirm.php" as s2
|
||||
s2 --> s3
|
||||
state "Host activates link" as s3
|
||||
state "Slot is reserved for a longer time" as s3
|
||||
state "upload.php" as s3
|
||||
s3 --> s4
|
||||
state "Host uploads show" as s4
|
||||
state "Host is notified upload complete" as s4
|
||||
state "upload_confirm.php" as s4
|
||||
s4 --> s5
|
||||
state "Dave Processes the shownotes" as s5
|
||||
s5 --> s6
|
||||
state "Ken Processes the media" as s6
|
||||
state "hprtranscode.bash" as s6
|
||||
state "Show posted to HPR" as s6
|
||||
s6 --> s7
|
||||
state "Dave posts to Archive.org" as s7
|
||||
s7 --> [*]
|
||||
```
|
||||
|
||||
# Getting Shows
|
||||
The upload process requires that the host reserve a slot, after which a link is sent to their email. Once they have the email link they can upload the show. The host then fills in the information related to their show. Some of the information is for processing the show and some is for display on the web site.
|
||||
|
||||
[request.php](request) > [request_confirm.php](request) > [upload.php](upload.php) > [upload_confirm.php](upload_confirm.php)
|
||||
|
||||
|
||||
# Processing Shows
|
||||
|
||||
The show notes (and any images) are copied and processed locally in order to generate HTML for adding to the database. This process is described under [Processing show notes](processing-shownotes).
|
||||
|
||||
The show media is downloaded locally and processed using [hprtranscode.bash](hprtranscode.bash).
|
||||
|
||||
# Posting Shows
|
||||
|
||||
# HPR DB Design
|
||||
|
||||
Currently this is the HPR DB Relationships for the HPR website.
|
||||
|
||||
|
||||
|
||||
BIN
mysql_db.jpg
Normal file
BIN
mysql_db.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 46 KiB |
67
request.md
Normal file
67
request.md
Normal file
@ -0,0 +1,67 @@
|
||||
# Requesting a slot
|
||||
|
||||
This process involves requesting a slot and verifying that the request is valid.
|
||||
|
||||
## Sequence Diagram
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
host->>HPR: Reserve Slot
|
||||
HPR->>HPR: Remove any old stale requests
|
||||
HPR->>HPR: Remove any requests from this IP older than 15min
|
||||
HPR->>HPR: Check for existing upload from this IP
|
||||
alt Existing Upload found
|
||||
HPR->>host: You already made a request for a show
|
||||
Note over host: 412 Precondition Failed
|
||||
end
|
||||
HPR->>HPR: Create a temporary entry for this host
|
||||
HPR->>HPR: Check for excessive uploads
|
||||
alt Excessive uploads found
|
||||
HPR->>host: Uploads have temporarily been suspended
|
||||
Note over host: 412 Precondition Failed
|
||||
end
|
||||
HPR->>HPR: Populate the list of posted shows
|
||||
HPR->>HPR: Validate provided ID
|
||||
alt Invalid ID found
|
||||
HPR->>HPR: Log IP to naughty file.
|
||||
HPR->>host: 412 Precondition Failed
|
||||
Note over host: 412 Precondition Failed
|
||||
end
|
||||
HPR->>HPR: Generate webform of free slots
|
||||
HPR->>HPR: Populate the list of posted shows
|
||||
HPR->>host: Webform
|
||||
host->>HPR: show ID and email
|
||||
loop Preform Checks
|
||||
alt If check fails
|
||||
HPR->>HPR: Log Error
|
||||
HPR->>host: 412 ${error_code}
|
||||
else All checks passed
|
||||
HPR-->>email: upload url
|
||||
email->>host: upload url
|
||||
end
|
||||
end
|
||||
```
|
||||
## Checks in request_confirm.php
|
||||
|
||||
| 412 Error Code | Check |
|
||||
| --- | ------ |
|
||||
| 5971624889258aefb44e5f7bf8dffbd4 | We are under DDOS attack. |
|
||||
| 19e9019c9615f755aec834000892ee9e | Wrong method used |
|
||||
| 9bb147a251e8db132dafa93d98f8487f | Your hiding your IP Address |
|
||||
| 02de1aef3b9490a417c39170d8f06028 | You did not use the web form |
|
||||
| 2162941738512bfdb1d21f288ee7cdb4 | You skipped the request page |
|
||||
| f0ad965f523b5c2ade071eb20d3618b5 | A breach in the space time continuum |
|
||||
| 6570026fd11fc31ac0cada3e1dae4d0b | There is too long a time entering the form |
|
||||
| a32fbe5f0494eb7f34034b164739314d | Wrong date |
|
||||
| 76eaa1a1556faeadfc14631c35b8590a | Missing an email address |
|
||||
| 8c307efe37146015a35e2d928c2c0f69 | Not a valid email |
|
||||
| 705f8e26e42a90b31075a110674b19ee | Not a valid date |
|
||||
| ad7f805c2f42be77122ec52f114fe318 | Date failed format check |
|
||||
| 9424f7407b2fb83407760ad763286b53 | Episode Number is wrong |
|
||||
| 59c7bff340d023773d987d71df545110 | Invalid date |
|
||||
| 47d186ad8d5b21ec7d455477ea08b023 | Episode already exists |
|
||||
| 7304801e8ce3b9096d28dbe1a0faa642 | Episode number is outside allowable window |
|
||||
| 34c4259b45927da50ba5c49970f880a4 | Episode date is outside allowable window |
|
||||
| d0e113355b35f96945124d8e507759a0 | Problem finding the date and episode number |
|
||||
| 434cb53552ce1e2708e74a42f438028c | Problem mapping the date to the episode number |
|
||||
| c7405e79b54f582e8db46c69ec4b0f24 | Problem writing to the database |
|
||||
1
test.md
Normal file
1
test.md
Normal file
@ -0,0 +1 @@
|
||||
Welcome to the Wiki.
|
||||
1
upload.md
Normal file
1
upload.md
Normal file
@ -0,0 +1 @@
|
||||
[request.php](request.php) > [request_confirm.php](request_confirm.php) > [upload.php](upload.php) > [upload_confirm.php](upload_confirm.php)
|
||||
1
upload_confirm.php.md
Normal file
1
upload_confirm.php.md
Normal file
@ -0,0 +1 @@
|
||||
[request.php](request.php) > [request_confirm.php](request_confirm.php) > [upload.php](upload.php) > [upload_confirm.php](upload_confirm.php)
|
||||
Loading…
Reference in New Issue
Block a user