jknapp
6d4ae2e5c3
- OAuth authentication via Authentik - WebSocket connection to OpenClaw gateway - Configurable gateway URL with first-run setup - User preferences sync across devices - Multi-user support with custom assistant names - ElevenLabs TTS integration (local + remote) - FCM push notifications for alarms - Voice input via Google Speech API - No hardcoded secrets or internal IPs in tracked files
39 KiB
39 KiB
Alfred Mobile Architecture
Complete system architecture showing authentication, networking, and communication flow
Note: This document uses example IPs (10.0.1.x) and domains (example.com) for privacy. Replace with your actual values when deploying.
System Overview
┌─────────────────────────────────────────────────────────────────────────┐
│ MOBILE DEVICE │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ Alfred Mobile App │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ │
│ │ │ AuthUI │ │ GatewayClient│ │ SharedPreferences │ │ │
│ │ │ (WebView) │ │ (WebSocket) │ │ - OAuth tokens │ │ │
│ │ │ │ │ │ │ - Token expiry │ │ │
│ │ └──────┬───────┘ └──────┬───────┘ └──────────────────────┘ │ │
│ │ │ │ │ │
│ └─────────┼─────────────────┼─────────────────────────────────────┘ │
│ │ │ │
└────────────┼─────────────────┼───────────────────────────────────────────┘
│ │
│ HTTPS │ WSS (WebSocket Secure)
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ INTERNET / NETWORK │
└─────────────────────────────────────────────────────────────────────────┘
│ │
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ HAProxy (10.0.1.20) │
│ ┌────────────────────────┐ │
│ │ SSL Termination │ │
│ │ - alfred-app. │ │
│ │ example.com:443 │ │
│ └────────┬───────────────┘ │
│ │ │
│ │ HTTP/WS (no SSL) │
│ │ │
└─────────────────────────────┼────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Windows Desktop (10.0.1.100) │
│ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ OAuth Proxy (Node.js) │ │
│ │ Port: 18790 (LAN-accessible) │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────┐ │ │
│ │ │ HTTP Endpoint: /api/notify │ │ │
│ │ │ - Accepts notification POST requests │ │ │
│ │ │ - Broadcasts to connected WebSocket clients │ │ │
│ │ └──────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────┐ │ │
│ │ │ WebSocket Handler │ │ │
│ │ │ 1. Validates OAuth token with Authentik │ │ │
│ │ │ 2. Connects to OpenClaw gateway (localhost) │ │ │
│ │ │ 3. Injects OpenClaw token into messages │ │ │
│ │ │ 4. Proxies bidirectional traffic │ │ │
│ │ │ 5. Tracks clients for notification broadcasting │ │ │
│ │ └──────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Connected Clients Set: │ │
│ │ • Tracks active WebSocket connections │ │
│ │ • Used for broadcast notifications │ │
│ └────────┬─────────────────────────────────────┬──────────────────┘ │
│ │ │ │
│ │ Validates │ Connects │
│ │ OAuth token │ (localhost only) │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────┐ ┌────────────────────────────┐ │
│ │ Authentik │ │ OpenClaw Gateway │ │
│ │ (10.0.1.75) │ │ Port: 18789 │ │
│ │ OAuth2 Provider │ │ Bind: loopback (127.0.0.1)│ │
│ │ │ │ Token: 9b87d1... │ │
│ │ - Client ID │ │ │ │
│ │ - Validates tokens │ │ ┌──────────────────────┐ │ │
│ │ - Returns userinfo │ │ │ Session: main │ │ │
│ └─────────────────────┘ │ │ - Chat interface │ │ │
│ │ │ - Agent tools │ │ │
│ │ └──────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────┐ │ │
│ │ │ mobile-notify skill │ │ │
│ │ │ - Sends to proxy │ │ │
│ │ └──────────────────────┘ │ │
│ └────────────────────────────┘ │
│ │
└───────────────────────────────────────────────────────────────────────────┘
Authentication Flow
┌──────────────┐
│ Mobile App │
│ (Starts) │
└──────┬───────┘
│
│ 1. No token? Launch OAuth flow
│
▼
┌──────────────────────────────────────────────────────────────┐
│ AuthUI (WebView) │
│ https://auth.example.com/application/o/authorize/ │
│ ?client_id=YOUR_OAUTH_CLIENT_ID │
│ &redirect_uri=alfredmobile://oauth/callback │
│ &response_type=code │
│ &scope=openid profile email │
└──────┬───────────────────────────────────────────────────────┘
│
│ 2. User logs in via Authentik
│
▼
┌──────────────────────────────────────────────────────────────┐
│ Authentik OAuth Provider (10.0.1.75) │
│ - Validates credentials │
│ - Issues authorization code │
│ - Redirects: alfredmobile://oauth/callback?code=AUTH_CODE_EXAMPLE │
└──────┬───────────────────────────────────────────────────────┘
│
│ 3. App captures redirect
│
▼
┌──────────────────────────────────────────────────────────────┐
│ AuthManager.handleCallback() │
│ - Extracts authorization code │
│ - Exchanges code for tokens │
└──────┬───────────────────────────────────────────────────────┘
│
│ 4. POST to token endpoint
│
▼
┌──────────────────────────────────────────────────────────────┐
│ POST https://auth.example.com/application/o/token/ │
│ { │
│ grant_type: "authorization_code", │
│ code: "AUTH_CODE_EXAMPLE", │
│ client_id: "QeSNa...", │
│ redirect_uri: "alfredmobile://oauth/callback" │
│ } │
└──────┬───────────────────────────────────────────────────────┘
│
│ 5. Receives tokens
│
▼
┌──────────────────────────────────────────────────────────────┐
│ Token Response: │
│ { │
│ access_token: "YOUR_JWT_TOKEN...", │
│ token_type: "Bearer", │
│ expires_in: 3600, │
│ refresh_token: "YOUR_REFRESH_TOKEN...", │
│ id_token: "YOUR_JWT_TOKEN..." │
│ } │
└──────┬───────────────────────────────────────────────────────┘
│
│ 6. Store in SharedPreferences
│
▼
┌──────────────────────────────────────────────────────────────┐
│ SharedPreferences (Android) │
│ - Key: "access_token" → "YOUR_JWT_TOKEN..." │
│ - Key: "token_expiry" → 1707089234567 (timestamp) │
│ - Key: "refresh_token" → "YOUR_REFRESH_TOKEN..." │
│ │
│ Storage location: /data/data/com.openclaw.alfred/ │
│ shared_prefs/auth_prefs.xml │
└──────┬───────────────────────────────────────────────────────┘
│
│ 7. Token valid? Connect to gateway
│
▼
┌──────────────────────────────────────────────────────────────┐
│ GatewayClient.connect() │
│ - WebSocket: wss://alfred-app.example.com │
│ - Header: Authorization: Bearer YOUR_JWT_TOKEN... │
└───────────────────────────────────────────────────────────────┘
WebSocket Communication Flow
┌──────────────┐
│ Mobile App │
│ (Connected) │
└──────┬───────┘
│
│ Send message
│
▼
┌────────────────────────────────────────────────────────────┐
│ WebSocket Message (WSS) │
│ wss://alfred-app.example.com │
│ │
│ { │
│ type: "req", │
│ id: "chat-1", │
│ method: "chat.send", │
│ params: { │
│ sessionKey: "main", │
│ message: "What's the weather?" │
│ } │
│ } │
└──────┬─────────────────────────────────────────────────────┘
│
│ TLS encrypted
│
▼
┌────────────────────────────────────────────────────────────┐
│ HAProxy (10.0.1.20) │
│ - Terminates TLS │
│ - Forwards to: http://10.0.1.100:18790 │
└──────┬─────────────────────────────────────────────────────┘
│
│ Plain HTTP/WS
│
▼
┌────────────────────────────────────────────────────────────┐
│ OAuth Proxy (10.0.1.100:18790) │
│ │
│ 1. Extract OAuth token from Authorization header │
│ Token: YOUR_JWT_TOKEN... │
│ │
│ 2. Validate with Authentik │
│ GET https://auth.example.com/application/o/userinfo/ │
│ Authorization: Bearer YOUR_JWT_TOKEN... │
│ │
│ 3. Authentik responds with user info │
│ { sub: "...", email: "user@...", ... } │
│ │
│ 4. Valid? Connect to OpenClaw gateway │
│ ws://127.0.0.1:18789 │
│ │
│ 5. Proxy receives message from mobile app │
│ │
│ 6. Inject OpenClaw token │
│ If message is connect request: │
│ params.auth.token = "9b87d15f..." │
│ │
│ 7. Forward to OpenClaw │
└──────┬─────────────────────────────────────────────────────┘
│
│ Localhost only
│
▼
┌────────────────────────────────────────────────────────────┐
│ OpenClaw Gateway (127.0.0.1:18789) │
│ │
│ 1. Receives message with injected token │
│ │
│ 2. Validates OpenClaw token │
│ Token: 9b87d15f... ✓ │
│ │
│ 3. Routes to agent session "main" │
│ │
│ 4. Agent processes message │
│ Message: "What's the weather?" │
│ │
│ 5. Agent generates response │
│ Response: "Currently 54°F in Your City..." │
│ │
│ 6. Sends chat event back │
│ { │
│ type: "event", │
│ event: "chat", │
│ payload: { │
│ message: { role: "assistant", content: [...] } │
│ } │
│ } │
└──────┬─────────────────────────────────────────────────────┘
│
│ Response flows back through proxy
│
▼
┌────────────────────────────────────────────────────────────┐
│ OAuth Proxy │
│ - Receives response from OpenClaw │
│ - Forwards to mobile app (pass-through) │
└──────┬─────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────┐
│ HAProxy │
│ - Wraps in TLS │
│ - Sends to mobile device │
└──────┬─────────────────────────────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────────┐
│ Mobile App │
│ - GatewayClient.onMessage() │
│ - Updates UI: "Currently 54°F in Your City..." │
└────────────────────────────────────────────────────────────┘
Notification Flow (mobile-notify tool)
┌────────────────────────────────────────────────────────────┐
│ Alfred Agent (OpenClaw) │
│ │
│ User says: "Remind me to check the laundry in 15 minutes" │
│ │
│ Agent executes: │
│ exec(['mobile-notify', 'timer', '15m', │
│ 'Check the laundry']) │
└──────┬─────────────────────────────────────────────────────┘
│
│ Shell execution
│
▼
┌────────────────────────────────────────────────────────────┐
│ mobile-notify CLI tool │
│ ~/.openclaw/workspace/skills/mobile-notify/ │
│ │
│ 1. Parse command: timer "15m" "Check the laundry" │
│ │
│ 2. For timers/reminders: Schedule via cron │
│ POST http://localhost:18789/api/cron/add │
│ { │
│ schedule: { kind: "at", atMs: 1707089234567 }, │
│ payload: { │
│ kind: "agentTurn", │
│ message: "Execute: mobile-notify alert ..." │
│ } │
│ } │
│ │
│ 3. For instant alerts: Send immediately │
│ POST http://localhost:18790/api/notify │
└──────┬─────────────────────────────────────────────────────┘
│
│ HTTP POST (for instant alerts)
│
▼
┌────────────────────────────────────────────────────────────┐
│ OAuth Proxy: /api/notify endpoint │
│ │
│ { │
│ notificationType: "alert", │
│ title: "Alfred", │
│ message: "Check the laundry", │
│ priority: "default", │
│ sound: true, │
│ vibrate: true │
│ } │
│ │
│ 1. Create notification event │
│ 2. Broadcast to all connected WebSocket clients │
│ connectedClients.forEach(client => { │
│ client.send(JSON.stringify({ │
│ type: "event", │
│ event: "mobile.notification", │
│ payload: {...} │
│ })) │
│ }) │
└──────┬─────────────────────────────────────────────────────┘
│
│ WebSocket broadcast
│
▼
┌────────────────────────────────────────────────────────────┐
│ All Connected Mobile Clients │
│ │
│ GatewayClient.handleEvent() │
│ event: "mobile.notification" │
│ → GatewayListener.onNotification() │
│ │
│ MainScreen.onNotification() │
│ 1. Add icon based on type (⏰ ⚠️ 🔔) │
│ 2. Show system notification (background OR timer/remind) │
│ NotificationHelper.showNotification() │
│ 3. Add to chat if foreground │
│ 4. Optional TTS if enabled │
└────────────────────────────────────────────────────────────┘
Token Storage Locations
Mobile App (Android)
File: /data/data/com.openclaw.alfred/shared_prefs/auth_prefs.xml
<map>
<string name="access_token">eyJhbGciOiJSUzI1NiIs...</string>
<long name="token_expiry">1707089234567</long>
<string name="refresh_token">YOUR_REFRESH_TOKEN...</string>
</map>
Access: AuthManager.kt
- getAccessToken()
- isTokenExpired()
- refreshToken()
OAuth Proxy (Memory)
Location: In-memory Set (JavaScript)
const connectedClients = new Set();
Each WebSocket connection:
- OAuth token validated once on connect
- Connection stored in Set for broadcasts
- Removed from Set on disconnect
OpenClaw Gateway
Location: ~/.openclaw/config.json
{
"gateway": {
"token": "YOUR_OPENCLAW_TOKEN",
"port": 18789,
"bind": "loopback"
}
}
Used by: OAuth Proxy only (localhost connection)
Authentik (Database)
Location: PostgreSQL database (10.0.1.75)
Tables:
- oauth2_provider_accesstoken
- oauth2_provider_refreshtoken
- oauth2_provider_authorizationcode
Validates tokens via userinfo endpoint
Network Topology
Internet
│
└─── Router/Firewall (10.0.1.1)
│
├─── HAProxy VM (10.0.1.20:443)
│ - SSL termination
│ - Reverse proxy
│
├─── Authentik (10.0.1.75:443)
│ - OAuth2 provider
│ - User authentication
│
└─── Windows Desktop (10.0.1.100)
- WSL Ubuntu 22.04
│
├─── OAuth Proxy (port 18790)
│ - LAN-accessible
│ - Validates & proxies
│
└─── OpenClaw Gateway (port 18789)
- Localhost-only
- Main agent session
Security Model
Defense in Depth
-
Mobile App
- OAuth tokens stored in encrypted SharedPreferences
- Token expiry validation (30s buffer)
- Automatic refresh on expiry
-
Transport Security
- TLS 1.3 encryption (mobile → HAProxy)
- Valid SSL certificate
- HTTPS/WSS only for external connections
-
HAProxy
- SSL termination
- Rate limiting (if configured)
- IP filtering (if configured)
-
OAuth Proxy
- Validates OAuth token on every connection
- No token caching (validates with Authentik)
- Localhost-only connection to OpenClaw
- Injects OpenClaw token (never exposed to client)
-
OpenClaw Gateway
- Bind: loopback (127.0.0.1 only)
- Token authentication required
- Not directly accessible from network
-
Authentik
- OAuth2 standard compliance
- Secure token generation
- User session management
Token Security
OAuth Token (Mobile ↔ Proxy)
- Short-lived (1 hour)
- Refresh token rotation
- Validated on every connection
- Stored securely on device
OpenClaw Token (Proxy ↔ Gateway)
- Static token (configured)
- Never leaves localhost
- Never sent to mobile clients
- Injected by proxy
File Locations
Mobile App
~/.openclaw/workspace/alfred-mobile/
├── app/src/main/java/com/openclaw/alfred/
│ ├── auth/AuthManager.kt (OAuth handling)
│ ├── gateway/GatewayClient.kt (WebSocket client)
│ ├── ui/screens/MainScreen.kt (UI + notification handler)
│ ├── notifications/NotificationHelper.kt
│ └── storage/ConversationStorage.kt
├── ARCHITECTURE.md (this file)
└── secrets.properties (OAuth client ID)
OAuth Proxy
~/.openclaw/workspace/alfred-proxy/
├── server.js (Main proxy logic)
├── .env (Config + secrets)
└── /tmp/alfred-proxy.log (Runtime logs)
OpenClaw
~/.openclaw/
├── config.json (Gateway token + config)
└── workspace/skills/mobile-notify/ (Notification tool)
├── mobile-notify (CLI wrapper)
└── scripts/notify.js (Implementation)
System Services
/etc/systemd/system/alfred-proxy.service (if using systemd)
/tmp/alfred-proxy.log (proxy logs)
Ports Summary
| Service | Port | Bind | Access | Protocol |
|---|---|---|---|---|
| HAProxy | 443 | 0.0.0.0 | External | HTTPS/WSS |
| Authentik | 443 | 10.0.1.75 | LAN | HTTPS |
| OAuth Proxy | 18790 | 0.0.0.0 | LAN | HTTP/WS |
| OpenClaw Gateway | 18789 | 127.0.0.1 | Localhost | HTTP/WS |
URLs
| Purpose | URL |
|---|---|
| Mobile app connection | wss://alfred-app.example.com |
| OAuth authorize | https://auth.example.com/application/o/authorize/ |
| OAuth token | https://auth.example.com/application/o/token/ |
| OAuth userinfo | https://auth.example.com/application/o/userinfo/ |
| OAuth redirect | alfredmobile://oauth/callback |
| Proxy health check | http://10.0.1.100:18790/health |
| Proxy notify API | http://10.0.1.100:18790/api/notify |
Key Design Decisions
-
Two-token architecture
- OAuth token: Mobile app authentication
- OpenClaw token: Backend service authentication
- Separation of concerns + security
-
Proxy pattern
- Mobile app never has direct OpenClaw access
- Token injection at proxy layer
- Enables centralized validation
-
Localhost-only OpenClaw
- Reduces attack surface
- Proxy is single point of entry
- Gateway not exposed to network
-
Notification broadcasting
- All clients receive notifications
- Supports multiple devices
- Real-time push via WebSocket
-
SSL termination at HAProxy
- Centralized certificate management
- Backend uses plain HTTP (trusted LAN)
- Standard reverse proxy pattern
Version
1.0.0 - Initial architecture (February 2026)