Fix Fourthwall parsing breakage, add JSON-LD/sitemap sources, rework caching
Create Release / build (push) Successful in 5s

Fourthwall moved the product title from <h2> to <h1>, which silently emptied
[fourthwall_single] and [fourthwall_random]: both gated rendering on a title
match that could no longer succeed. Verified against a live store.

Parsing
- Read schema.org JSON-LD first, falling back to scraped markup field by field,
  then to og: meta tags. Each field degrades independently, so a future markup
  change can only affect what it actually touched.
- Match CSS classes with a padded contains() predicate instead of @class="...",
  and select on data-testid where available. Exact class matching is what broke.
- Surface sku, availability and currency, which were previously discarded.

Product discovery
- Enumerate products from sitemap.xml rather than scraping the store page, which
  only sees the collection it features. [fourthwall_random count="10"] could not
  return more than the 3 items a featured homepage renders.
- New source="auto|sitemap|page" attribute. Off-host <loc> entries are rejected
  so a hostile sitemap cannot redirect fetches.

Caching
- Stale-while-revalidate: stale entries are served immediately and refreshed by
  WP-Cron, so cache expiry never costs a visitor a network round trip.
- Refresh scheduling dedupes via wp_next_scheduled and takes a lock, collapsing
  a stampede to a single job.
- Fetch uncached product pages concurrently with curl_multi instead of serially.
- Cache lifetime is now configurable (default 60 minutes).
- Entries written by earlier versions are read as stale and upgraded in place,
  so caches turn over without blanking a store.

Fixes
- Build absolute product links properly. "/collections/all" + "/products/x"
  produced 404s; a trailing slash produced "//products/x".
- Escape the price on output; it was interpolated raw.
- strpos() misuse meant an error string at offset 0 was treated as success.
- Guard a null description node that would fatal when show_description="true".
- Report non-200 responses instead of caching an empty body silently.
- Drop the shared /tmp/cookies.txt jar. It was cross-site mutable state and made
  concurrent fetching unsafe; all endpoints return 200 without it.
- Remove the ssl_verify setting. It was a development affordance and its
  checkbox never worked - unchecked meant "key absent", which read as true.
  Certificate verification is now pinned on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-06 09:33:06 -07:00
co-authored by Claude Opus 5
parent f22615c6d5
commit ecef4d3f2e
4 changed files with 1094 additions and 165 deletions
+25 -14
View File
@@ -30,6 +30,14 @@ Displays a random selection of products from your store.
- `count` (optional): Number of products to display (default: 3)
- `urls` (optional): Comma-separated list of specific product URLs to randomize from
- `store_url` (optional): Custom store URL (uses default from settings if not provided)
- `source` (optional): Where to discover products - `auto` (default), `sitemap`, or `page`
**About `source`:** products are discovered from your store's `sitemap.xml`, which
lists your entire catalog. The older behaviour scraped the store page, which only
sees the collection that page happens to feature - so if your homepage shows 3
featured items, `count="10"` could never return more than 3. `auto` uses the
sitemap and falls back to scraping the page if the sitemap is unavailable; use
`page` to force the old behaviour.
#### Examples:
@@ -58,11 +66,16 @@ Displays a random selection of products from your store.
In the WordPress admin, go to Settings > Fourthwall Store Embed to set your Fourthwall store URL.
**Cache Lifetime** (default: 60 minutes) controls how often store content is
refreshed. Content is cached and served instantly; once it passes the lifetime it
is *still* served immediately while a background refresh runs, so raising or
lowering this never slows a page down. Use **Clear Cache** to force an immediate
refresh after changing products in Fourthwall.
## Features
- Caches requests for better performance
- Responsive design
- SSL verification options
- Error handling for failed requests
- Random product selection
- Support for multiple store URLs
@@ -77,12 +90,10 @@ In the WordPress admin, go to Settings > Fourthwall Store Embed to set your Four
**Store URL**: Enter your Fourthwall store URL (e.g., `https://your-store.fourthwall.com`)
**SSL Verification**:
- **Enabled (Recommended)**: Use for production sites to ensure secure connections
- **Disabled**: Use only for local development when SSL certificates are not properly configured
**Cache Lifetime**: How long store content stays fresh, in minutes (default: 60)
**Cache Management**:
- Content is automatically cached for 1 hour to improve performance
- Content is cached and served instantly, then refreshed in the background
- Use the "Clear Cache" button if products are not updating
#### Display your entire store
@@ -98,8 +109,8 @@ You can also display the product description by setting the `show_description` a
### Features
- **Smart Caching**: Automatic caching system reduces server load and improves performance
- **Configurable SSL**: Toggle SSL verification for development vs production environments
- **Smart Caching**: Stale content is served instantly while it refreshes in the background, so cache expiry never slows a page down
- **Parallel Fetching**: Uncached products are fetched concurrently rather than one at a time
- **Error Handling**: Graceful fallbacks and clear error messages
- **Admin Interface**: User-friendly settings page with clear instructions
- **Cache Management**: Manual cache clearing for troubleshooting
@@ -113,21 +124,21 @@ You can also display the product description by setting the `show_description` a
### Performance Notes
- Content is cached for 1 hour to reduce API calls to Fourthwall
- Cache automatically refreshes when content changes
- Content is cached to reduce requests to Fourthwall (default: 60 minutes)
- Once the cache lifetime passes, content is still served instantly while a
background refresh runs, so visitors never wait on the network
- Manual cache clearing available in admin settings
- SSL verification can be disabled for local development
### Troubleshooting
**Products not updating?**
- Clear the cache using the "Clear Cache" button in admin settings
- Check your store URL is correct
- Verify SSL verification setting matches your environment
- Lower the Cache Lifetime if you need changes to appear sooner
**SSL errors in development?**
- Disable SSL verification in admin settings (development only)
- Ensure proper SSL certificates in production
**Background refreshes not happening?**
- Refreshes run via WP-Cron. If `DISABLE_WP_CRON` is set with no system cron
configured, content is served stale until it is refetched on demand.
**403 Forbidden errors?**
- Fourthwall may be blocking automated requests