Fix Fourthwall parsing breakage, add JSON-LD/sitemap sources, rework caching

Browse Source

Fourthwall moved the product title from <h2> to <h1>, which silently emptied
[fourthwall_single] and [fourthwall_random]: both gated rendering on a title
match that could no longer succeed. Verified against a live store.

Parsing
- Read schema.org JSON-LD first, falling back to scraped markup field by field,
  then to og: meta tags. Each field degrades independently, so a future markup
  change can only affect what it actually touched.
- Match CSS classes with a padded contains() predicate instead of @class="...",
  and select on data-testid where available. Exact class matching is what broke.
- Surface sku, availability and currency, which were previously discarded.

Product discovery
- Enumerate products from sitemap.xml rather than scraping the store page, which
  only sees the collection it features. [fourthwall_random count="10"] could not
  return more than the 3 items a featured homepage renders.
- New source="auto|sitemap|page" attribute. Off-host <loc> entries are rejected
  so a hostile sitemap cannot redirect fetches.

Caching
- Stale-while-revalidate: stale entries are served immediately and refreshed by
  WP-Cron, so cache expiry never costs a visitor a network round trip.
- Refresh scheduling dedupes via wp_next_scheduled and takes a lock, collapsing
  a stampede to a single job.
- Fetch uncached product pages concurrently with curl_multi instead of serially.
- Cache lifetime is now configurable (default 60 minutes).
- Entries written by earlier versions are read as stale and upgraded in place,
  so caches turn over without blanking a store.

Fixes
- Build absolute product links properly. "/collections/all" + "/products/x"
  produced 404s; a trailing slash produced "//products/x".
- Escape the price on output; it was interpolated raw.
- strpos() misuse meant an error string at offset 0 was treated as success.
- Guard a null description node that would fatal when show_description="true".
- Report non-200 responses instead of caching an empty body silently.
- Drop the shared /tmp/cookies.txt jar. It was cross-site mutable state and made
  concurrent fetching unsafe; all endpoints return 200 without it.
- Remove the ssl_verify setting. It was a development affordance and its
  checkbox never worked - unchecked meant "key absent", which read as true.
  Certificate verification is now pinned on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

...

This commit is contained in:

shadowdao

2026-08-06 09:33:06 -07:00

co-authored by Claude Opus 5

parent f22615c6d5

commit ecef4d3f2e

4 changed files with 1094 additions and 165 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

libs/shortcode.php

+901 -131

File diff suppressed because it is too large Load Diff