rho_n
/
hpr_generator
4
2
Fork 1
Code Issues 34 Pull Requests Projects 1 Releases Wiki Activity

Problems with Unicode in the site-generator #158

New Issue
Closed
opened 2023-08-27 10:28:58 +00:00 by davmo · 2 comments
davmo commented 2023-08-27 10:28:58 +00:00
Collaborator
Copy Link

The current version of the site-generator (2023-08-27) produces badly encoded Unicode in shows which have such characters. There are currently 603 such shows with Unicode in titles, summaries, tags and notes.

The problem might have been introduced in the last update, and seems to be related to the addition of tag parsing code.

The current version of the `site-generator` (2023-08-27) produces badly encoded Unicode in shows which have such characters. There are currently 603 such shows with Unicode in titles, summaries, tags and notes. The problem might have been introduced in the last update, and seems to be related to the addition of tag parsing code.
davmo commented 2023-08-27 12:48:29 +00:00
Author
Collaborator
Copy Link

Created a pull request which undoes the Unicode (utf8) adjustments from the last update and defaults to all utf8 IO. These updates had been added to try and stop a problem with tags containing utf8 characters, but did not work.

The latest change detects utf8 characters in tags and ensures they are properly encoded. This happens in subroutine parse_csv which provides the vmethod csv_parse.

Using a clone of the static site, 60 shows were chosen at random from the 603 shows containing utf8, and these were checked. Only one showed encoding problems and that seems to be because of invalid characters in the database.

Created a pull request which undoes the Unicode (`utf8`) adjustments from the last update and defaults to all `utf8` IO. These updates had been added to try and stop a problem with tags containing `utf8` characters, but did not work. The latest change detects `utf8` characters in tags and ensures they are properly encoded. This happens in subroutine `parse_csv` which provides the `vmethod` `csv_parse`. Using a clone of the static site, 60 shows were chosen at random from the 603 shows containing `utf8`, and these were checked. Only one showed encoding problems and that seems to be because of invalid characters in the database.
davmo commented 2023-08-27 13:56:37 +00:00
Author
Collaborator
Copy Link

Pull request #159 merged.

Closing this issue.

Pull request #159 merged. Closing this issue.
davmo closed this issue 2023-08-27 13:56:37 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
167_Several_templates_use_non-existent_absolute_url
I145_Various_bug_fixes
I123_Future_episodes_are_not_being_produced
I30-Series_name_for_filename
2023-03
2023-02
Labels
Clear labels   
Bug Found

For issues related to the code or the content of the created HTML files

  
Feature Request

   
General Discussion

For discussing HPR site generation in general

  
Help Request

For issues running or installing the site-generator

No Label
Bug Found
Feature Request
General Discussion
Help Request
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: rho_n/hpr_generator#158
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?