Commit Graph
73 Commits
Author SHA1 Message Date
shadowdao f8c5a6a1be Merge: memory edit scope/project change (Agent A) 2026-05-17 06:48:48 -07:00
shadowdaoandClaude Opus 4.7 fcc8708dd5 feat(snippets): named reusable artifacts with scope mirroring memories
Snippets are exact-name-keyed templates (PR formats, checklists, style
rules) — prescriptive artifacts the user wants applied consistently.
Distinct from memories, which are searched descriptive prose.

- Migration 0002 adds scope/project_id/deleted_at to snippets, partial
  unique indexes per scope, and a scope/project CHECK constraint
  mirroring memories_scope_project_chk.
- New @shared-memory/schemas: SnippetName/Put/Get/List/Delete inputs
  with shared scope-project refinement.
- lib/snippets.ts: get/put/list/softDelete helpers used by both the
  Web Server Actions and the MCP tool handlers.
- Four new MCP tools (snippet.put/get/list/delete) with directive
  descriptions contrasting against memory.* (exact-name lookup vs
  search; templates vs facts).
- /snippets pages: list, new, [name] with edit + delete and a
  scope-picker when a name lives in multiple scopes.
- Nav: Snippets link between Memories and Projects.

Design note: when a name exists in both user and project scope,
snippet.get without an explicit scope prefers project (if project key
given) then falls back to user. The detail page shows a picker when
the name is ambiguous and no scope query param was supplied.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 06:48:00 -07:00
shadowdao d65e1cb351 feat(memory): allow editing scope and project on memory.update
Extends both the Web UI edit form and the memory.update MCP tool so
existing memories can be reclassified between user-global and
project-attached scopes without delete+rewrite. Schema refines enforce
the user/project consistency invariants; audit log captures from/to
scope and projectKey on transitions.
2026-05-17 06:43:19 -07:00
shadowdaoandClaude Opus 4.7 af5e7c4680 fix(web): remove stray onSubmit on delete form
Server components can't pass function props (onSubmit handlers etc.) to
client-rendered HTML elements. The handler here was a no-op leftover —
removing it fixes the 500 'Event handlers cannot be passed to Client
Component props' on /memories/[id].

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 11:56:49 -07:00
shadowdaoandClaude Opus 4.7 2e5c81c0ee docs(mcp): clarify that sensitive info IS appropriate for shared-memory
Per-user OIDC-gated storage is strictly safer than writing API keys /
credentials to local container files, so the tool description should
not discourage that use case. Adds an explicit allowlist for sensitive
data the user actively shares (vs. asking for them).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 11:29:58 -07:00
shadowdaoandClaude Opus 4.7 76d0159b3d feat(mcp): directive tool descriptions to drive proactive use
The Phase 1/2 descriptions explained WHAT each tool does. The model has
no other signal for WHEN to call them, so they only fire on explicit
user prompts ("remember that…", "search for…"). This rewrites each
description to lead with the trigger condition.

Highlights:
- memory.write now explicitly contrasts with the built-in file-based
  memory at ~/.claude/.../memory/, so transient or container-specific
  state stays there while user/project facts go here
- memory.search description tells the model to call it BEFORE answering
  any question that might touch a previously-shared fact ("cheap; lean
  toward calling it")
- project.identify becomes a session-start ritual when there's a repo
  context, anchoring all subsequent project-scoped writes
- update vs delete: explicit "prefer update" guidance to keep ids stable

No code or schema changes — descriptions are pure metadata that surface
in the MCP tools/list response on the next session of every connected
client.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 11:27:11 -07:00
shadowdaoandClaude Opus 4.7 ff6baab393 feat: Phase 3b — proper Web UI for memories, projects, settings
Replaces the debug /me + /connect pages with a real authed app shell.

Pages
- /                       — anonymous landing; redirects to /dashboard once signed in
- /dashboard              — recent memories + top projects, quick "new memory" action
- /memories               — searchable list with hybrid (vector+FTS+tags) scoring;
                            per-result rank breakdown shown inline
- /memories/[id]          — view + inline edit toggle + delete
- /memories/new           — create form with project autocomplete
- /projects               — list with memory counts and last-activity
- /projects/[key]         — that project's memories
- /settings               — read-only Authentik profile + link to tokens
- /settings/tokens        — list / create / revoke CLI tokens

Old URLs preserved as redirects:
- /me      → /dashboard
- /connect → /settings/tokens

Stack additions
- Tailwind v4 with CSS-first @theme tokens (dark only for now)
- App shell in app/(authed)/ — auth guard + top nav with global search box
- Lightweight UI primitives in app/_components/ui/ (Button, Input, Card,
  Badge, EmptyState, Container, PageHeader)
- Search logic extracted from MCP tool into lib/memories.ts so Web UI and
  MCP both call the same RRF code path
- Memory CRUD via Server Actions in lib/memory-actions.ts; audit_log
  rows are tagged actor='web' to distinguish from MCP writes

Per-token revoke
- New cli_tokens table (id, user_id, jti unique, name, created_at,
  last_used_at, expires_at, revoked_at) — migration 0001_cli_tokens.sql
- mintCliToken now records jti + name; verifyCliToken enforces revocation
  for tracked tokens. Legacy tokens minted before this change (no jti)
  are accepted on signature alone until they expire naturally.
- /settings/tokens lists active + revoked tokens with one-click revoke

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 10:57:17 -07:00
shadowdaoandClaude Opus 4.7 609039f098 feat: /auth/cli-callback page + OAuth-first connect docs
Adds the manual-paste fallback page MCP clients hit when their OAuth
loopback callback isn't reachable (sealed containers, port-restricted
hosts). The page displays the authorization code, the full callback URL,
and the state parameter, all with copy buttons, plus instructions to
paste back into the waiting terminal. Single-use codes plus client-side
PKCE mean displaying the code here is safe — it isn't a credential by
itself.

README "Connecting Claude Code" rewritten to make OAuth the primary
path, with three options ordered by preference: (A) standard OAuth +
loopback, (B) manual paste via /auth/cli-callback, (C) static HMAC
bearer via /connect for fully headless setups. Also notes that the
zero-config plugin path is blocked on Authentik DCR (issue #8751,
expected later this year) so we're shipping the one-liner now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 10:37:08 -07:00
shadowdaoandClaude Opus 4.7 d7707dd158 refactor: generic OIDC provider — Auth.js no longer Authentik-specific
Replaces the Authentik-preset provider with an inline generic OIDC config
(id: "oidc"). The verifier path was already protocol-generic; the only
Authentik-named piece was the next-auth provider preset, which mapped to
the same OIDC fields anyway. Any compliant IdP — Authentik, EntraID,
Keycloak, Okta, Auth0, Zitadel, etc. — now works with just the existing
OIDC_* env vars.

Breaking change for existing deployments: the Auth.js callback path
changes from /api/auth/callback/authentik → /api/auth/callback/oidc.
Update the Web-UI client's redirect URI in your IdP before redeploying.

README rewritten to frame Authentik as the worked example, with a
concept-mapping table for EntraID and Keycloak, and audience-claim notes
for non-Authentik IdPs in the troubleshooting section.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 10:15:44 -07:00
shadowdaoandClaude Opus 4.7 10a1e0062e fix(embedder): switch base image to node:20-slim
onnxruntime-node ships glibc-linked native binaries; loading them on
node:20-alpine (musl libc) fails with:

  Error loading shared library ld-linux-x86-64.so.2: No such file or
  directory (needed by …/onnxruntime-node/bin/napi-v3/linux/x64/
  libonnxruntime.so.1.14.0)

Using node:20-slim (Debian) provides glibc and the dynamic linker the
native bindings expect. Image is ~50 MB larger but actually works.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 09:13:36 -07:00
shadowdaoandClaude Opus 4.7 a6f944b44d fix(docker): use pnpm deploy for embedder runtime image
The previous Dockerfile copied apps/embedder/node_modules straight from
the deps stage, but with pnpm's default isolated layout that directory
is a farm of relative symlinks pointing into the root .pnpm store. With
the root node_modules absent in the runtime image, every dependency
resolution failed (fastify, @xenova/transformers, ...).

pnpm deploy --prod writes a portable directory with flat node_modules
that resolves cleanly from a fresh WORKDIR. The `files` field on the
embedder package opts dist/ into the deployed output (the workspace
.gitignore would otherwise exclude it).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 09:09:40 -07:00
shadowdaoandClaude Opus 4.7 9a8b504f51 feat: Phase 2 — embeddings + hybrid memory.search
Adds a small embedder sidecar (Xenova/bge-small-en-v1.5, ONNX, CPU-only)
that the web app calls inline on memory.write and memory.update, and on
demand from the new memory.search tool.

memory.search performs three candidate fetches in parallel — pgvector
cosine similarity, Postgres full-text via plainto_tsquery + ts_rank_cd,
and tag-set overlap — then fuses them with Reciprocal Rank Fusion
(k=60). Each result carries its per-source rank so the model can see
*why* a memory surfaced.

The migrator boot step gained an idempotent embedding backfill: any row
with embedding IS NULL is batched (32 at a time) through the embedder
after SQL migrations apply. Safe to run on every boot.

New tool memory.update fixes the missing edit path; centralises the
re-embed-on-content-change rule alongside write.

Stack additions:
- apps/embedder/ — Fastify server, persistent /data/models volume so the
  ~30 MB model only downloads once
- apps/web/lib/embedder.ts — typed HTTP client with batched embed +
  health probe
- packages/schemas — MemoryUpdateInput, MemorySearchInput
- docker-compose — embedder service, healthcheck, app + migrator both
  depend_on it healthy; EMBEDDER_URL promoted to a required env var

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 09:04:44 -07:00
shadowdaoandClaude Opus 4.7 e1fa1197f4 fix(compose): pass CLI_TOKEN_SECRET through to the app service
The explicit `environment:` map in compose filters which .env vars reach
the container. Adding CLI_TOKEN_SECRET so the new CLI-token verifier can
read it at runtime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 08:38:05 -07:00
shadowdaoandClaude Opus 4.7 2bd8ea705d feat(auth): CLI tokens minted at /connect for containerized MCP clients
Adds a second token kind alongside Authentik OIDC access tokens for MCP
authentication. When the user visits /connect after signing into the Web
UI, the server mints an HMAC-signed JWT (kid="cli-v1") carrying their
Authentik identity in oidc_iss / oidc_sub claims. The token is shown
once in React state — never put in the URL or persisted on the client.

The MCP endpoint's bearer-token verifier dispatches by JWT `kid` header:
CLI tokens are verified locally via HS256(CLI_TOKEN_SECRET); everything
else goes through Authentik JWKS. Both paths resolve to the same
AuthenticatedClaims shape so userContextFromClaims handles them
identically.

This unblocks MCP clients running in containers where the OAuth loopback
callback isn't reachable — paste the token into Claude Code as a static
Authorization header and skip the OAuth flow entirely.

Revocation in v1 is "rotate CLI_TOKEN_SECRET to invalidate every issued
CLI token at once." Per-token revocation can come later if needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 08:36:11 -07:00
shadowdaoandClaude Opus 4.7 7ab6dfe6f6 fix(auth): remove self-referential pages.signIn config
pages.signIn is meant to point at a CUSTOM sign-in page. Setting it to
'/api/auth/signin' — Auth.js's own built-in endpoint — makes Auth.js
redirect there whenever it wants the sign-in page, which is the same
endpoint, producing ERR_TOO_MANY_REDIRECTS in browsers.

Omitting the setting falls back to Auth.js's default sign-in handler,
which renders the provider-picker HTML at /api/auth/signin instead of
redirecting.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 08:15:41 -07:00
shadowdaoandClaude Opus 4.7 3de1701175 fix(env): coerce empty EMBEDDER_URL to undefined
Zod's .url().optional() still rejects "" because the empty string is a
present value. EMBEDDER_URL is unused in Phase 1 and intentionally left
blank in .env, so preprocess "" to undefined before validation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 08:10:47 -07:00
shadowdaoandClaude Opus 4.7 7218e69b1d fix(migration): drop invalid trigram index on tags[] column
gin_trgm_ops only applies to text, not text[]. The plain GIN index on
tags is sufficient for the `@>` / `<@` / `&&` set-containment operators
used by memory.list. pg_trgm stays loaded for future fuzzy search over
content.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 08:08:22 -07:00
shadowdaoandClaude Opus 4.7 973cc68832 fix(migrate): resolve migrations dir alongside or above the script
When esbuild bundles migrate.ts to apps/web/migrate.mjs (one directory
higher than the source location), the relative `..` path in the previous
implementation pointed at apps/, not apps/web/. Try both layouts and
allow an explicit MIGRATIONS_DIR override.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 08:06:06 -07:00
shadowdaoandClaude Opus 4.7 d5b7206bda fix(docker): use package-relative paths for esbuild bundle step
pnpm --filter runs in the package directory, so the input and output
paths must be relative to apps/web, not the repo root.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 08:03:14 -07:00
shadowdaoandClaude Opus 4.7 3b191c449f fix(docker): declare esbuild as direct dev dependency
The Dockerfile uses `pnpm exec esbuild` to bundle the migrator into a
single ESM file before copying it into the runtime image. pnpm exec only
resolves binaries from declared dependencies, so the transitive esbuild
that tsx pulls in wasn't visible to the build stage.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:59:11 -07:00
shadowdaoandClaude Opus 4.7 d4cd53ad2b fix(docker): copy packages/schemas/node_modules in builder stage
Local pnpm builds worked because all workspace packages had their
node_modules populated by `pnpm install`. The Docker builder stage was
only restoring `apps/web/node_modules` from the deps stage, leaving
`packages/schemas/node_modules` empty — so `next build` couldn't resolve
`zod` when transpiling the shared schemas package.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:55:29 -07:00
shadowdaoandClaude Opus 4.7 077d0a0825 feat: Phase 1 — Authentik auth, MCP endpoint, persistent memory
End-to-end Phase 1 of shared-memory: a logged-in Authentik user can sign
into the Web UI (/me debug page), and an MCP client with an Authentik-
issued bearer token can call memory.write / memory.list / memory.get /
memory.delete plus project.identify against /api/mcp.

Stack:
- Next.js 15 (App Router) + React 19 + TypeScript, pnpm workspaces
- Drizzle ORM + Postgres 16 + pgvector + pg_trgm
- Auth.js v5 with Authentik provider (Web UI)
- jose + Authentik JWKS for MCP bearer-token validation
- JSON-RPC 2.0 dispatcher implementing the MCP wire protocol over plain
  HTTP POST (hand-rolled to fit Next.js App Router; switches to SSE in a
  later phase if server-initiated events are needed)
- bge-small embeddings sidecar deferred to Phase 2; the schema already
  reserves the vector(384) column + IVFFlat index, FTS via a STORED
  tsvector column, and the visibility enum (private/shared/team) so
  cross-user memory sharing can be added without a future migration

Deployment supports two modes (set in .env, never committed):
- Behind an external reverse proxy (HAProxy / nginx / Cloudflare Tunnel /
  Traefik) — DEFAULT; the app exposes APP_PORT on the host with
  X-Forwarded-* trusted, no in-container TLS
- Built-in TLS via Caddy — opt-in with `docker compose --profile tls up`

Discovery endpoint at /.well-known/oauth-protected-resource (RFC 9728)
points MCP clients at the Authentik authorization server after a 401.

README walks through both Authentik providers (Web UI + MCP resource
server), the audience scope mapping, redirect URIs, and includes a worked
HAProxy config snippet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 07:04:11 -07:00
shadowdaoandClaude Opus 4.7 d5be753cfb Initial commit: gitignore and env example
Establishes the secrets-handling contract for this repo: real env values
live in a local .env (gitignored from the first commit), and only the
sanitized .env.example with placeholder values is tracked.

.env.example documents the env surface for the v1 deployment: PUBLIC_URL,
Authentik OIDC clients (web + MCP resource server), Postgres connection,
embedder sidecar, NextAuth secret, and log level.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 06:46:18 -07:00