Feed updates

feedWatcher

@@ -21,14 +21,17 @@
 #               (created with Pandoc) in the same directory as this script for
 #               details of what has been done to develop the original design.
 #
+#               Further development is taking place in 2022/2023, again for
+#               FOSDEM.
+#
 #      OPTIONS: ---
 # REQUIREMENTS: ---
 #         BUGS: ---
 #        NOTES: ---
 #       AUTHOR: Dave Morriss (djm), Dave.Morriss@gmail.com
-#      VERSION: 0.1.1
+#      VERSION: 0.1.2
 #      CREATED: 2013-12-25 12:40:33
-#     REVISION: 2023-01-09 15:28:13
+#     REVISION: 2023-01-10 22:44:38
 #
 #-------------------------------------------------------------------------------
 # Released under the terms of the GNU Affero General Public License (AGPLv3)
@@ -44,6 +47,10 @@ use utf8;
 use feature qw{ postderef say signatures state };
 no warnings qw{ experimental::postderef experimental::signatures } ;
 
+#
+# There's an issue in XML::RSS, so we're using a loocal version with a hack.
+# It's in ./lib/ and FiindBin::libs looks there to find it.
+#
 use FindBin::libs;
 use XML::RSS;
 
@@ -85,7 +92,7 @@ use Data::Dumper;
 #
 # Version number (manually incremented)
 #
-our $VERSION = '0.1.1';
+our $VERSION = '0.1.2';
 
 #
 # Script name
@@ -2538,7 +2545,7 @@ feedWatcher - watch a collection of podcast feeds
 
 =head1 VERSION
 
-This documentation refers to I<feedWatcher> version 0.1.1
+This documentation refers to I<feedWatcher> version 0.1.2
 
 
 =head1 USAGE
@@ -2547,10 +2554,12 @@ This documentation refers to I<feedWatcher> version 0.1.1
     [-check[=mode]] [-out=FILE] [-json[=FILE]] [-opml[=FILE]] [-template=FILE]
     [-[no]silent] [-config=FILE] [-debug=N] [URL ...]
 
- # Load URLs from a file, perform checks and redirect output to standard
+ # Load URLs from a file, perform checks, save the rejects and redirect output
- # output and a named file
+ # to a named file. Uses an alias:
- ./feedWatcher -load=feedWatcher_dumped_URLs.txt -check=auto | \
+ # alias isostamp='date +"%Y%m%d_%H%M%S"'
-    tee load_$(date +'%Y%m%d_%H%M%S')
+ #
+ ./feedWatcher -load=feedWatcher_dumped_URLs.txt -check=auto \
+     -rej=output/rejects_$(isostamp).out > output/load_$(isostamp).out 2>&1
 
  # Generate Markdown output with a template writing to a named file
  ./feedWatcher -tem=feedWatcher_3.tpl -out=feedWatcher.mkd
@@ -2607,26 +2616,69 @@ script, (B<-noscan>) omits the scan.
 
 NOTE: This function is not implemented yet.
 
+=item B<-[no]silent>
+
+This option controls the amount of output written by the script. In
+B<nosilent> mode the script reports on the processing of each URL it receives,
+which can be fairly verbose. This can be turned off with this option, though
+it is often wiser to redirect the output for later review rather than to
+suppress it.
+
 =item B<-out=FILE>
 
-This option defines an output file to receive outputi from reporting
+This option defines an output file to receive output from the reporting
 functions. If the option is omitted the data is written to STDOUT, allowing it
-to be redirected if required. This option does not cause transactional
+to be redirected if required. See the 'Usage' section above for an example of
-listings to be captured.
+how transactional output can be redirected.
 
-=item B<-[no]check>
+=item B<-check[=MODE]>
 
-This option (B<-check>) causes each feed which is being to be checked against
+This option (B<-check[=MODE]>) controls the mode used to check
-the script user to check that it's OK to add it. The script reports the
+the copyright setting of the current feed and deciide whether to add it.
-I<copyright> field and requests a I<y> or I<n> response.
 
-=item B<-[no]report>
+Possible settings are: B<auto>, B<manual> and B<none>.
 
-This option (B<-report>) causes a report of the contents of the database to be
+=over 4
-generated.  The negated form, which is also the default behaviour of the
-script, (B<-noreport>) omits the report.
 
-NOTE: The report is currently very simple.
+=item B<-check=auto> or B<-check>
+
+An automatic check is made against a series of regular expressions looking for
+something in the I<copyright> field which signifies that the feed is under
+a Creative Commons licence. A blank field is currently considered to denote
+this type of licence.
+
+The option may be written as B<-check> when it is interpreted as B<-check=auto>
+
+=item B<-check=manual>
+
+In this mode the script pauses after processing each feed to ask the script
+user to check that it's OK to add it. The script reports the I<copyright>
+field and requests a I<y> or I<n> response.
+
+=item B<-check=manual>
+
+In this mode no checks are performed.
+
+=back
+
+=item B<-report[=title]>
+
+This option (B<-report[=title]>) causes a fairly simplistic report to be
+generated to enable the database contents to be examined. The I<title>
+argument specifies a case-sensitive feed title or component of such a title.
+So, for instance B<-report=Hacker> currently reports on the batabase data
+relating to the "Hacker Public Radio" feed.
+
+If the argument is omitted the entire database is reported.
+
+Reports consist of the details of the RSS (or Atom) channel with other
+information about the site hosting the feed such as the IP address. The latest
+episode in the feed is also reported.
+
+Note that the feed information in the database is a snapshot made when the
+feed details were last loaded. This is static information and does not get
+updatedunless the feed is deleted and reloaded, or the B<-scan> function is run
+(not currently available).
 
 =item B<-json[=FILE]>
 
@@ -2650,10 +2702,10 @@ file.
 If the B<=FILE> portion is omitted a default name of 'feedWatcher.opml' is
 used.
 
-=item B<-template=FILE>
+=item B<-template[=FILE]>
 
 This option defines the template used to generate a form of the feed data. The
-template is written using the B<Template> toolkit language.
+template is written using the B<Template Toolkit> language.
 
 If the file name is omitted then the script uses the file B<feedWatcher.tpl>
 in the same directory as the script. If this file does not exist then the

feedWatcher.html

@@ -406,6 +406,14 @@
         
     
         
+            <dt><a href="https://www.eff.org/how-to-fix-the-internet-podcast">How to Fix the Internet</a> (<a href="https://feeds.eff.org/howtofixtheinternet">feed</a>)</dt>
+        
+        
+            <dd>The internet is broken&mdash;but it doesn&rsquo;t have to be. If you&rsquo;re concerned about how surveillance, online advertising, and automated content moderation are hurting us online and offline, the Electronic Frontier Foundation&rsquo;s How to Fix the Internet podcast offers a better way forward. EFF has been defending your rights online for over thirty years and is behind many of the biggest digital rights protections since the invention of the internet. Through curious conversations with some of the leading minds in law and technology, this podcast explores creative solutions to some of today&rsquo;s biggest tech challenges. Hosted by EFF Executive Director Cindy Cohn and EFF Associate Director of Digital Strategy Jason Kelley, How to Fix the Internet will help you become deeply informed on vital technology issues as we work to build a better technological future together.</dd>
+        
+        
+    
+        
             <dt><a href="http://internationalopenmagazine.org/category/podcast.html">International Open Podcast</a> (<a href="http://spielend-programmieren.at/intopenpodcast.xml">feed</a>)</dt>
         
         

feedWatcher.json

@@ -987,6 +987,44 @@
       "urls_url" : "http://hackerpublicradio.org/hpr_ogg_rss.php",
       "urls_urltype" : "Feed"
    },
+   {
+      "ep_author" : "podcast@eff.org (Cindy Cohn, Jason Kelley)",
+      "ep_byte_length" : 2639064,
+      "ep_category" : "",
+      "ep_enclosure" : "https://cdn.simplecast.com/audio/1c515ea8-cb6d-4f72-8d17-bc9b7a566869/episodes/ec20459c-105b-4c05-a18d-2e6334ab5c66/audio/b697806f-a61d-4ad7-9b58-f510d1c7733b/default_tc.mp3?aid=rss_feed&feed=r0ih__Uy",
+      "ep_ep_id" : "de24d72c-7ec9-42a8-a236-1212b0330c19",
+      "ep_id" : 6435,
+      "ep_issued" : "2023-01-09 08:10:00",
+      "ep_last_update" : "2023-01-11 09:41:10",
+      "ep_link" : "https://www.eff.org/how-to-fix-the-internet-podcast",
+      "ep_mime_type" : "audio/mpeg",
+      "ep_modified" : null,
+      "ep_source" : null,
+      "ep_title" : "Coming Soon: How to Fix the Internet Season 4",
+      "ep_urls_id" : 97,
+      "id" : 97,
+      "latest_ep" : "2023-01-09 08:10:00",
+      "urls_author" : null,
+      "urls_child_count" : 0,
+      "urls_content_type" : "text/xml; charset=UTF-8",
+      "urls_copyright" : null,
+      "urls_description" : "The internet is broken—but it doesn’t have to be. If you’re concerned about how surveillance, online advertising, and automated content moderation are hurting us online and offline, the Electronic Frontier Foundation’s How to Fix the Internet podcast offers a better way forward. EFF has been defending your rights online for over thirty years and is behind many of the biggest digital rights protections since the invention of the internet. Through curious conversations with some of the leading minds in law and technology, this podcast explores creative solutions to some of today’s biggest tech challenges. Hosted by EFF Executive Director Cindy Cohn and EFF Associate Director of Digital Strategy Jason Kelley, How to Fix the Internet will help you become deeply informed on vital technology issues as we work to build a better technological future together.",
+      "urls_dns" : "5.135.16.50",
+      "urls_feedformat" : "RSS 2.0",
+      "urls_generator" : "https://simplecast.com",
+      "urls_host_up" : 1,
+      "urls_http_status" : "200 OK",
+      "urls_id" : 97,
+      "urls_image" : "https://image.simplecastcdn.com/images/6d0021c7-4504-4cd8-a194-c8d6c2b4fd7a/14e8afb5-8608-4501-8d90-c3e5b4551078/3000x3000/podcast-square-season-4-temp.jpg?aid=rss_feed",
+      "urls_language" : "en",
+      "urls_last_update" : "2023-01-11 09:41:10",
+      "urls_link" : "https://www.eff.org/how-to-fix-the-internet-podcast",
+      "urls_modified" : "2023-01-09 08:10:00",
+      "urls_parent_id" : null,
+      "urls_title" : "How to Fix the Internet",
+      "urls_url" : "https://feeds.eff.org/howtofixtheinternet",
+      "urls_urltype" : "Feed"
+   },
    {
       "ep_author" : null,
       "ep_byte_length" : 120830454,

feedWatcher.mkd

@@ -136,6 +136,11 @@
     - Feed: http://hackerpublicradio.org/hpr_ogg_rss.php
     - Licence: Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License
 
+- **How to Fix the Internet**
+    - Website: https://www.eff.org/how-to-fix-the-internet-podcast
+    - Feed: https://feeds.eff.org/howtofixtheinternet
+    - Licence: 
+
 - **International Open Podcast**
     - Website: http://internationalopenmagazine.org/category/podcast.html
     - Feed: http://spielend-programmieren.at/intopenpodcast.xml

feedWatcher.opml

@@ -2,8 +2,8 @@
 <opml version="1.1">
 <head>
 <title>Free Culture Podcasts</title>
-<dateCreated>2023-01-10 19:43:26</dateCreated>
+<dateCreated>2023-01-11 09:44:06</dateCreated>
-<dateModified>2023-01-10 19:43:26</dateModified>
+<dateModified>2023-01-11 09:44:06</dateModified>
 <ownerName></ownerName>
 <ownerEmail></ownerEmail>
 <expansionState></expansionState>
@@ -40,6 +40,7 @@
 <outline description="GeekNights Mondays is the weekly sci/tech segment of GeekNights, featuring science, technology, computing, and more. We talk Linux, Windows, gadgets, you name it." htmlUrl="http://frontrowcrew.com/geeknights/monday/" text="GeekNights Mondays: Science Technology Computing" title="GeekNights Mondays: Science Technology Computing" xmlUrl="http://feeds.feedburner.com/GNSciTech"  />
 <outline description="Once you become aware that there is a dependable, secure, capable, and modern computer system that rivals all others in popularity and actual use, you will want to try the Linux operating system on your computer. Perhaps you've been using a member of the Unix/Linux family - Linux, Android, ChromeOS, BSD or even OSX - for quite a while. If so, you are likely looking for new ways to optimize your technology for the way you work. Going Linux is for computer users who just want to use Linux to get things done. Are you new to Linux, upgrading from Windows to Linux, or just thinking about moving to Linux? This audio podcast provides you with practical, day-to-day advice on how to use Linux and its applications. Our goal is to help make the Linux experience easy for you." htmlUrl="https://goinglinux.com" text="Going Linux" title="Going Linux" xmlUrl="http://goinglinux.com/oggpodcast.xml"  />
 <outline description="Hacker Public Radio is an podcast that releases shows every weekday Monday through Friday. Our shows are produced by the community (you) and can be on any topic that are of interest to hackers and hobbyists." htmlUrl="http://hackerpublicradio.org/about.php" text="Hacker Public Radio" title="Hacker Public Radio" xmlUrl="http://hackerpublicradio.org/hpr_ogg_rss.php"  />
+<outline description="The internet is broken—but it doesn’t have to be. If you’re concerned about how surveillance, online advertising, and automated content moderation are hurting us online and offline, the Electronic Frontier Foundation’s How to Fix the Internet podcast offers a better way forward. EFF has been defending your rights online for over thirty years and is behind many of the biggest digital rights protections since the invention of the internet. Through curious conversations with some of the leading minds in law and technology, this podcast explores creative solutions to some of today’s biggest tech challenges. Hosted by EFF Executive Director Cindy Cohn and EFF Associate Director of Digital Strategy Jason Kelley, How to Fix the Internet will help you become deeply informed on vital technology issues as we work to build a better technological future together." htmlUrl="https://www.eff.org/how-to-fix-the-internet-podcast" text="How to Fix the Internet" title="How to Fix the Internet" xmlUrl="https://feeds.eff.org/howtofixtheinternet"  />
 <outline description="The podcast of the makers of international open magazine" htmlUrl="http://internationalopenmagazine.org/category/podcast.html" text="International Open Podcast" title="International Open Podcast" xmlUrl="http://spielend-programmieren.at/intopenpodcast.xml"  />
 <outline description="The most recent additions to the Internet Archive collections. This RSS feed is generated dynamically" htmlUrl="https://archive.org/" text="Internet Archive" title="Internet Archive" xmlUrl="http://feeds.feedburner.com/archive/Lqwl"  />
 <outline description="The cross platform podcast that makes technology work for you and not the other way around. The place to go for all geeks who slide between Mac, iOS, Android, Linux and Windows offering an essential mix of hacks, tips, howto's and tweaks spiced up with a dash of geek culture. Also check out our Mediafeed that has both our audio and video episodes." htmlUrl="https://knightwise.com" text="Knightwise.com Audio Feed." title="Knightwise.com Audio Feed." xmlUrl="http://feeds.feedburner.com/knightcastpodcast"  />