Merge pull request 'Skip AUR for now, attach Arch package as a GitHub release asset' (#42) from fix/aur-render-expression-collision into main

Reviewed-on: #42

.gitea/workflows/publish-arch-package.yml

@@ -1,37 +1,36 @@
-name: Publish AUR Package
+name: Publish Arch Package
 
-# Builds and pushes the `triple-c-bin` AUR package (packaging/arch/PKGBUILD)
+# Builds the `triple-c-bin` Arch package (packaging/arch/PKGBUILD) for a
-# for a given release, or the latest one if none is given. Manual dispatch
+# given release, or the latest one if none is given, and attaches the built
-# only — deliberately not triggered by `release` or `push`, for the same
+# .pkg.tar.zst to that release on GitHub as a downloadable asset. Manual
-# reason sync-release.yml (removed in triple-c#32) never worked safely as an
+# dispatch only — deliberately not triggered by `release` or `push`, for the
-# automatic trigger: this repo's releases are assembled by build-app.yml
+# same reason sync-release.yml (removed in triple-c#32) never worked safely
-# across three separate platform jobs, and there is no single automatic event
+# as an automatic trigger: this repo's releases are assembled by
-# that fires only once everything (including the Linux .deb this workflow
+# build-app.yml across three separate platform jobs, and there is no single
-# needs) is actually uploaded. A human deciding "this release is ready, go
+# automatic event that fires only once everything (including the Linux .deb
-# package it" is the correct trigger, the same reasoning
+# this workflow needs) is actually uploaded. A human deciding "this release
+# is ready, go package it" is the correct trigger, the same reasoning
 # backfill-releases.yml already uses for its own manual-only GitHub sync.
 #
 # ## What this does and does not do
 #
 # It renders `packaging/arch/PKGBUILD` for one specific version (real
-# download URL, real sha256sums — never guessed; see the resolve-asset step)
+# download URL, real sha256sums — never guessed; see the resolve-asset step),
-# and pushes the rendered PKGBUILD plus a regenerated `.SRCINFO` to AUR. It
+# validates it with `makepkg`/`namcap` in a real Arch container, and uploads
-# does NOT commit anything back to this repo — `packaging/arch/PKGBUILD` stays
+# the resulting `.pkg.tar.zst` to the GitHub release it was built from —
-# a hand-maintained template with a placeholder version, and every real,
+# installable by hand with `pacman -U`. It does NOT commit anything back to
-# published version lives only in AUR's own git history, which is where a
+# this repo — `packaging/arch/PKGBUILD` stays a hand-maintained template with
-# PKGBUILD's revision history is expected to live. A corollary worth knowing:
+# a placeholder version, and the workflow never starts from or writes to it.
-# a hand-edit made directly in the AUR repo (outside this workflow) is
-# silently overwritten the next time this runs, since every run renders fresh
-# from this repo's template rather than starting from AUR's current state.
 #
-# ## Required secret
+# ## Not published to the AUR (yet)
 #
-# `AUR_SSH_PRIVATE_KEY` — an SSH private key registered against an AUR
+# This originally also pushed the rendered PKGBUILD to an AUR git repo, which
-# account that has already created (or been given co-maintainer access to)
+# needs a maintainer AUR account and its SSH key registered as a secret here
-# the `triple-c-bin` package. This workflow cannot create that AUR account or
+# — both manual, one-time steps neither this workflow nor anyone but a
-# register the key for you — both are manual, one-time steps on
+# maintainer can do. Until that setup happens, a downloadable release asset
-# https://aur.archlinux.org. Until this secret exists, every run fails at the
+# gets the same package to users without it. The AUR push step is still in
-# "Push to AUR" step with a clear error rather than silently doing nothing.
+# this file's git history (see the commit that added this comment) if that
+# setup is ever done and it's worth reinstating.
 on:
   workflow_dispatch:
     inputs:
@@ -43,7 +42,6 @@ on:
 
 env:
   GITHUB_REPO: shadowdao/triple-c
-  AUR_REPO: ssh://aur@aur.archlinux.org/triple-c-bin.git
 
 jobs:
   publish:
@@ -97,10 +95,20 @@ jobs:
           fi
           echo "Found asset: ${DEB_NAME}"
 
+          # For attaching the built package to this same release later —
+          # every release object carries its own `upload_url` regardless of
+          # whether it was just created or (as here) already existed, and
+          # the `{?name,label}` URI-template suffix has to come off before
+          # this is usable as a plain URL to POST to.
+          RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
+          UPLOAD_URL=$(echo "$RELEASE_JSON" | jq -r '.upload_url' | sed 's/{?name,label}//')
+
           echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
           echo "tag=${TAG}" >> "$GITHUB_OUTPUT"
           echo "deb_url=${DEB_URL}" >> "$GITHUB_OUTPUT"
           echo "deb_name=${DEB_NAME}" >> "$GITHUB_OUTPUT"
+          echo "release_id=${RELEASE_ID}" >> "$GITHUB_OUTPUT"
+          echo "upload_url=${UPLOAD_URL}" >> "$GITHUB_OUTPUT"
 
       - name: Download the release asset and compute real checksums
         id: checksums
@@ -197,6 +205,7 @@ jobs:
           ! grep -q "SKIP" PKGBUILD
 
       - name: Validate with makepkg and namcap
+        id: build
         run: |
           set -euo pipefail
 
@@ -218,6 +227,13 @@ jobs:
             echo "builder ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/builder
             sudo -u builder bash -c "cd /work && makepkg --printsrcinfo > .SRCINFO"
             sudo -u builder bash -c "cd /work && makepkg -s --noconfirm"
+            # Named once here, inside the container, rather than guessed
+            # from options=(!strip !debug) plus pkgver/pkgrel/arch on the
+            # host after the fact — makepkg is the one place that actually
+            # knows its own output name, and `!debug` already guarantees
+            # this glob can only ever match the one real package (no
+            # -debug split package gets produced).
+            basename /work/*.pkg.tar.* > /work/.pkgfile
             echo "--- namcap ---"
             NAMCAP_OUT=$(sudo -u builder bash -c "cd /work && namcap PKGBUILD *.pkg.tar.*" || true)
             echo "$NAMCAP_OUT"
@@ -238,50 +254,45 @@ jobs:
           # bind mount still fails the job the same way.
           docker start -a "${CID}"
           docker cp "${CID}:/work/.SRCINFO" rendered/.SRCINFO
+          docker cp "${CID}:/work/.pkgfile" rendered/.pkgfile
+          PKG_FILE=$(cat rendered/.pkgfile)
+          docker cp "${CID}:/work/${PKG_FILE}" "rendered/${PKG_FILE}"
           docker rm -f "${CID}" >/dev/null
 
-      - name: Push to AUR
+          echo "pkg_file=${PKG_FILE}" >> "$GITHUB_OUTPUT"
+
+      - name: Attach the package to the GitHub release
         env:
-          AUR_SSH_PRIVATE_KEY: ${{ secrets.AUR_SSH_PRIVATE_KEY }}
+          GH_PAT: ${{ secrets.GH_PAT }}
-          VERSION: ${{ steps.resolve.outputs.version }}
+          TAG: ${{ steps.resolve.outputs.tag }}
+          RELEASE_ID: ${{ steps.resolve.outputs.release_id }}
+          UPLOAD_URL: ${{ steps.resolve.outputs.upload_url }}
+          PKG_FILE: ${{ steps.build.outputs.pkg_file }}
         run: |
           set -euo pipefail
-          if [ -z "${AUR_SSH_PRIVATE_KEY}" ]; then
+          if [ -z "${GH_PAT}" ]; then
-            echo "AUR_SSH_PRIVATE_KEY is not set — see this workflow file's header comment for" >&2
+            echo "GH_PAT is not set — this step needs it to attach a release asset." >&2
-            echo "the one-time AUR account setup this needs before it can publish anything." >&2
             exit 1
           fi
 
-          mkdir -p ~/.ssh
+          # A manual re-dispatch for a version that's already been packaged
-          # Created with the final mode before any bytes land in it, rather
+          # would otherwise hit GitHub's 422 "already_exists" here instead
-          # than a plain redirect followed by chmod, which leaves the key
+          # of just replacing the stale build with this one.
-          # world-readable for whatever window falls between the two calls.
+          EXISTING_ID=$(curl -fsS -H "Authorization: Bearer ${GH_PAT}" -H "Accept: application/vnd.github+json" \
-          install -m 600 /dev/null ~/.ssh/aur
+            "https://api.github.com/repos/${GITHUB_REPO}/releases/${RELEASE_ID}/assets" \
-          echo "${AUR_SSH_PRIVATE_KEY}" > ~/.ssh/aur
+            | jq -r --arg name "$PKG_FILE" '.[] | select(.name == $name) | .id')
-          # TOFU, not verification — accepted here because pinning AUR's
+          if [ -n "$EXISTING_ID" ]; then
-          # actual host key needs a value fetched from somewhere trusted
+            echo "Replacing the existing ${PKG_FILE} (asset id ${EXISTING_ID}) already on ${TAG}"
-          # ahead of time, which this workflow doesn't have, and getting a
+            curl -fsS -X DELETE -H "Authorization: Bearer ${GH_PAT}" -H "Accept: application/vnd.github+json" \
-          # pinned value wrong fails every future run rather than just this
+              "https://api.github.com/repos/${GITHUB_REPO}/releases/assets/${EXISTING_ID}"
-          # one. A keyscan failure below surfaces later as an opaque
-          # "Host key verification failed" rather than a clear one here.
-          ssh-keyscan -H aur.archlinux.org >> ~/.ssh/known_hosts 2>/dev/null
-          export GIT_SSH_COMMAND="ssh -i ~/.ssh/aur -o IdentitiesOnly=yes -o UserKnownHostsFile=~/.ssh/known_hosts"
-
-          git clone "${AUR_REPO}" aur-repo
-          cp rendered/PKGBUILD rendered/.SRCINFO aur-repo/
-          cd aur-repo
-          git config user.name "Triple-C CI"
-          git config user.email "noreply@triple-c.invalid"
-          git add PKGBUILD .SRCINFO
-          if git diff --cached --quiet; then
-            echo "No change from what's already published on AUR for ${VERSION}"
-            exit 0
           fi
-          git commit -m "triple-c-bin: update to ${VERSION}"
+
-          # AUR itself uses `master`, which is what a fresh, not-yet-created
+          curl -fsS -X POST \
-          # AUR package's empty repo advertises on clone — but the *local*
+            -H "Authorization: Bearer ${GH_PAT}" \
-          # branch name after cloning an empty repo falls back to whatever
+            -H "Accept: application/vnd.github+json" \
-          # this runner's `init.defaultBranch` is if the server sends no
+            -H "Content-Type: application/octet-stream" \
-          # symref, so naming the destination explicitly is what keeps this
+            --data-binary "@rendered/${PKG_FILE}" \
-          # working if that default is ever `main` instead of `master`.
+            "${UPLOAD_URL}?name=$(python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1]))" "${PKG_FILE}")" \
-          git push origin HEAD:master
+            > /dev/null
+
+          echo "Attached ${PKG_FILE} to ${TAG}"

TECHNICAL.md

@@ -418,10 +418,10 @@ triple-c/
 │       ├── build-stt.yml            # Build the STT image
 │       ├── backfill-releases.yml    # Bulk copy releases to GitHub
 │       ├── cleanup-releases.yml     # Prune old releases
-│       └── publish-aur-package.yml  # Publish triple-c-bin to the AUR (packaging/arch/)
+│       └── publish-arch-package.yml # Build triple-c-bin, attach it to the GitHub release (packaging/arch/)
 │
 ├── packaging/
-│   └── arch/                      # AUR triple-c-bin package — see packaging/arch/README.md
+│   └── arch/                      # triple-c-bin Arch package — see packaging/arch/README.md
 │       ├── PKGBUILD
 │       └── README.md
 │

packaging/arch/PKGBUILD

@@ -1,6 +1,6 @@
 # Maintainer: Triple-C Contributors
 #
-# This file is regenerated by .gitea/workflows/publish-aur-package.yml on every
+# This file is regenerated by .gitea/workflows/publish-arch-package.yml on every
 # publish — pkgver, the source URL and sha256sums are rewritten from the real,
 # already-uploaded release asset, never guessed. Editing pkgver/source/
 # sha256sums by hand here only matters until the next automated run overwrites

packaging/arch/README.md

@@ -1,7 +1,9 @@
 # Arch / CachyOS package
 
-`PKGBUILD` here is the AUR `triple-c-bin` package's template — see triple-c#34
+`PKGBUILD` here is the `triple-c-bin` package's template — see triple-c#34
 (the "I would like to also have an Arch/CachyOS native version" part of it).
+It's written to AUR conventions (and may go there eventually — see
+"Publishing" below) but isn't published to the AUR yet.
 
 ## Why "-bin"
 
@@ -18,24 +20,23 @@ to be real dependencies of *this* binary and were dropped), and ran a real
 
 ## Publishing
 
-`.gitea/workflows/publish-aur-package.yml` does the actual work: given a
+`.gitea/workflows/publish-arch-package.yml` does the actual work: given a
 version (or "latest" if none is given), it finds that release's real Linux
 asset on GitHub, downloads it, computes real checksums, renders this
 template into a version-specific PKGBUILD, validates it with `makepkg` and
-`namcap` inside a real Arch container, and pushes the result to AUR.
+`namcap` inside a real Arch container, and attaches the resulting
+`.pkg.tar.zst` to that same GitHub release as a downloadable asset —
+installable by hand with `sudo pacman -U`.
 
 It is `workflow_dispatch`-only, deliberately — see the workflow file's own
 header comment for why an automatic trigger isn't safe here (the same reason
 `sync-release.yml` didn't work and was removed in triple-c#32).
 
-**Before it can push anything**, an AUR account has to exist and the
+**Not on the AUR yet.** Publishing there would need a maintainer AUR account
-`triple-c-bin` package has to have been created (or you added as a
+and its SSH key added as a secret on this repo — both manual, one-time steps
-co-maintainer) under it — both are one-time, manual steps on
+on https://aur.archlinux.org that only a maintainer can do. The workflow's
-https://aur.archlinux.org, since there's no API to automate creating an
+git history still has the AUR-push step from before this was descoped, if
-account or a new package. Once that's done, add the account's SSH private
+that setup happens later and it's worth reinstating.
-key as the `AUR_SSH_PRIVATE_KEY` secret on this repo. Until that secret
-exists, the workflow fails at the "Push to AUR" step with a message saying
-so, rather than silently doing nothing.
 
 ## What's hand-maintained vs. generated
 
@@ -45,9 +46,7 @@ result back here, so don't read this file's `pkgver` as "the last published
 version." Everything else (`depends`, `pkgdesc`, `package()`) is meant to be
 edited by hand normally, the same as any other PKGBUILD.
 
-**A hand-edit made directly in the AUR repo is silently overwritten the
+**A hand-edit made to the rendered PKGBUILD attached to a GitHub release is
-next time this workflow runs.** Every run renders fresh from *this*
+not this file.** Every run renders fresh from *this* repo's template, so a
-repo's template rather than starting from whatever AUR's copy currently
+packaging fix belongs here, not in a downloaded copy — the next dispatch for
-looks like, so a quick fix pushed straight to AUR (bumping `pkgrel` for a
+that version would just overwrite it anyway.
-packaging-only issue, say) survives only until the next dispatch. Make
-the fix here instead.