Files
alfred-mobile/ARCHITECTURE.md

606 lines
39 KiB
Markdown
Raw Permalink Normal View History

# Alfred Mobile Architecture
**Complete system architecture showing authentication, networking, and communication flow**
> **Note:** This document uses example IPs (10.0.1.x) and domains (example.com) for privacy. Replace with your actual values when deploying.
## System Overview
```
┌─────────────────────────────────────────────────────────────────────────┐
│ MOBILE DEVICE │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ Alfred Mobile App │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ │
│ │ │ AuthUI │ │ GatewayClient│ │ SharedPreferences │ │ │
│ │ │ (WebView) │ │ (WebSocket) │ │ - OAuth tokens │ │ │
│ │ │ │ │ │ │ - Token expiry │ │ │
│ │ └──────┬───────┘ └──────┬───────┘ └──────────────────────┘ │ │
│ │ │ │ │ │
│ └─────────┼─────────────────┼─────────────────────────────────────┘ │
│ │ │ │
└────────────┼─────────────────┼───────────────────────────────────────────┘
│ │
│ HTTPS │ WSS (WebSocket Secure)
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ INTERNET / NETWORK │
└─────────────────────────────────────────────────────────────────────────┘
│ │
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ HAProxy (10.0.1.20) │
│ ┌────────────────────────┐ │
│ │ SSL Termination │ │
│ │ - alfred-app. │ │
│ │ example.com:443 │ │
│ └────────┬───────────────┘ │
│ │ │
│ │ HTTP/WS (no SSL) │
│ │ │
└─────────────────────────────┼────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ Windows Desktop (10.0.1.100) │
│ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ OAuth Proxy (Node.js) │ │
│ │ Port: 18790 (LAN-accessible) │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────┐ │ │
│ │ │ HTTP Endpoint: /api/notify │ │ │
│ │ │ - Accepts notification POST requests │ │ │
│ │ │ - Broadcasts to connected WebSocket clients │ │ │
│ │ └──────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────┐ │ │
│ │ │ WebSocket Handler │ │ │
│ │ │ 1. Validates OAuth token with Authentik │ │ │
│ │ │ 2. Connects to OpenClaw gateway (localhost) │ │ │
│ │ │ 3. Injects OpenClaw token into messages │ │ │
│ │ │ 4. Proxies bidirectional traffic │ │ │
│ │ │ 5. Tracks clients for notification broadcasting │ │ │
│ │ └──────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Connected Clients Set: │ │
│ │ • Tracks active WebSocket connections │ │
│ │ • Used for broadcast notifications │ │
│ └────────┬─────────────────────────────────────┬──────────────────┘ │
│ │ │ │
│ │ Validates │ Connects │
│ │ OAuth token │ (localhost only) │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────┐ ┌────────────────────────────┐ │
│ │ Authentik │ │ OpenClaw Gateway │ │
│ │ (10.0.1.75) │ │ Port: 18789 │ │
│ │ OAuth2 Provider │ │ Bind: loopback (127.0.0.1)│ │
│ │ │ │ Token: 9b87d1... │ │
│ │ - Client ID │ │ │ │
│ │ - Validates tokens │ │ ┌──────────────────────┐ │ │
│ │ - Returns userinfo │ │ │ Session: main │ │ │
│ └─────────────────────┘ │ │ - Chat interface │ │ │
│ │ │ - Agent tools │ │ │
│ │ └──────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────┐ │ │
│ │ │ mobile-notify skill │ │ │
│ │ │ - Sends to proxy │ │ │
│ │ └──────────────────────┘ │ │
│ └────────────────────────────┘ │
│ │
└───────────────────────────────────────────────────────────────────────────┘
```
## Authentication Flow
```
┌──────────────┐
│ Mobile App │
│ (Starts) │
└──────┬───────┘
│ 1. No token? Launch OAuth flow
┌──────────────────────────────────────────────────────────────┐
│ AuthUI (WebView) │
│ https://auth.example.com/application/o/authorize/ │
│ ?client_id=YOUR_OAUTH_CLIENT_ID │
│ &redirect_uri=alfredmobile://oauth/callback │
│ &response_type=code │
│ &scope=openid profile email │
└──────┬───────────────────────────────────────────────────────┘
│ 2. User logs in via Authentik
┌──────────────────────────────────────────────────────────────┐
│ Authentik OAuth Provider (10.0.1.75) │
│ - Validates credentials │
│ - Issues authorization code │
│ - Redirects: alfredmobile://oauth/callback?code=AUTH_CODE_EXAMPLE │
└──────┬───────────────────────────────────────────────────────┘
│ 3. App captures redirect
┌──────────────────────────────────────────────────────────────┐
│ AuthManager.handleCallback() │
│ - Extracts authorization code │
│ - Exchanges code for tokens │
└──────┬───────────────────────────────────────────────────────┘
│ 4. POST to token endpoint
┌──────────────────────────────────────────────────────────────┐
│ POST https://auth.example.com/application/o/token/ │
│ { │
│ grant_type: "authorization_code", │
│ code: "AUTH_CODE_EXAMPLE", │
│ client_id: "QeSNa...", │
│ redirect_uri: "alfredmobile://oauth/callback" │
│ } │
└──────┬───────────────────────────────────────────────────────┘
│ 5. Receives tokens
┌──────────────────────────────────────────────────────────────┐
│ Token Response: │
│ { │
│ access_token: "YOUR_JWT_TOKEN...", │
│ token_type: "Bearer", │
│ expires_in: 3600, │
│ refresh_token: "YOUR_REFRESH_TOKEN...", │
│ id_token: "YOUR_JWT_TOKEN..." │
│ } │
└──────┬───────────────────────────────────────────────────────┘
│ 6. Store in SharedPreferences
┌──────────────────────────────────────────────────────────────┐
│ SharedPreferences (Android) │
│ - Key: "access_token" → "YOUR_JWT_TOKEN..." │
│ - Key: "token_expiry" → 1707089234567 (timestamp) │
│ - Key: "refresh_token" → "YOUR_REFRESH_TOKEN..." │
│ │
│ Storage location: /data/data/com.openclaw.alfred/ │
│ shared_prefs/auth_prefs.xml │
└──────┬───────────────────────────────────────────────────────┘
│ 7. Token valid? Connect to gateway
┌──────────────────────────────────────────────────────────────┐
│ GatewayClient.connect() │
│ - WebSocket: wss://alfred-app.example.com │
│ - Header: Authorization: Bearer YOUR_JWT_TOKEN... │
└───────────────────────────────────────────────────────────────┘
```
## WebSocket Communication Flow
```
┌──────────────┐
│ Mobile App │
│ (Connected) │
└──────┬───────┘
│ Send message
┌────────────────────────────────────────────────────────────┐
│ WebSocket Message (WSS) │
│ wss://alfred-app.example.com │
│ │
│ { │
│ type: "req", │
│ id: "chat-1", │
│ method: "chat.send", │
│ params: { │
│ sessionKey: "main", │
│ message: "What's the weather?" │
│ } │
│ } │
└──────┬─────────────────────────────────────────────────────┘
│ TLS encrypted
┌────────────────────────────────────────────────────────────┐
│ HAProxy (10.0.1.20) │
│ - Terminates TLS │
│ - Forwards to: http://10.0.1.100:18790 │
└──────┬─────────────────────────────────────────────────────┘
│ Plain HTTP/WS
┌────────────────────────────────────────────────────────────┐
│ OAuth Proxy (10.0.1.100:18790) │
│ │
│ 1. Extract OAuth token from Authorization header │
│ Token: YOUR_JWT_TOKEN... │
│ │
│ 2. Validate with Authentik │
│ GET https://auth.example.com/application/o/userinfo/ │
│ Authorization: Bearer YOUR_JWT_TOKEN... │
│ │
│ 3. Authentik responds with user info │
│ { sub: "...", email: "user@...", ... } │
│ │
│ 4. Valid? Connect to OpenClaw gateway │
│ ws://127.0.0.1:18789 │
│ │
│ 5. Proxy receives message from mobile app │
│ │
│ 6. Inject OpenClaw token │
│ If message is connect request: │
│ params.auth.token = "9b87d15f..." │
│ │
│ 7. Forward to OpenClaw │
└──────┬─────────────────────────────────────────────────────┘
│ Localhost only
┌────────────────────────────────────────────────────────────┐
│ OpenClaw Gateway (127.0.0.1:18789) │
│ │
│ 1. Receives message with injected token │
│ │
│ 2. Validates OpenClaw token │
│ Token: 9b87d15f... ✓ │
│ │
│ 3. Routes to agent session "main" │
│ │
│ 4. Agent processes message │
│ Message: "What's the weather?" │
│ │
│ 5. Agent generates response │
│ Response: "Currently 54°F in Your City..." │
│ │
│ 6. Sends chat event back │
│ { │
│ type: "event", │
│ event: "chat", │
│ payload: { │
│ message: { role: "assistant", content: [...] } │
│ } │
│ } │
└──────┬─────────────────────────────────────────────────────┘
│ Response flows back through proxy
┌────────────────────────────────────────────────────────────┐
│ OAuth Proxy │
│ - Receives response from OpenClaw │
│ - Forwards to mobile app (pass-through) │
└──────┬─────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────┐
│ HAProxy │
│ - Wraps in TLS │
│ - Sends to mobile device │
└──────┬─────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────┐
│ Mobile App │
│ - GatewayClient.onMessage() │
│ - Updates UI: "Currently 54°F in Your City..." │
└────────────────────────────────────────────────────────────┘
```
## Notification Flow (mobile-notify tool)
```
┌────────────────────────────────────────────────────────────┐
│ Alfred Agent (OpenClaw) │
│ │
│ User says: "Remind me to check the laundry in 15 minutes" │
│ │
│ Agent executes: │
│ exec(['mobile-notify', 'timer', '15m', │
│ 'Check the laundry']) │
└──────┬─────────────────────────────────────────────────────┘
│ Shell execution
┌────────────────────────────────────────────────────────────┐
│ mobile-notify CLI tool │
│ ~/.openclaw/workspace/skills/mobile-notify/ │
│ │
│ 1. Parse command: timer "15m" "Check the laundry" │
│ │
│ 2. For timers/reminders: Schedule via cron │
│ POST http://localhost:18789/api/cron/add │
│ { │
│ schedule: { kind: "at", atMs: 1707089234567 }, │
│ payload: { │
│ kind: "agentTurn", │
│ message: "Execute: mobile-notify alert ..." │
│ } │
│ } │
│ │
│ 3. For instant alerts: Send immediately │
│ POST http://localhost:18790/api/notify │
└──────┬─────────────────────────────────────────────────────┘
│ HTTP POST (for instant alerts)
┌────────────────────────────────────────────────────────────┐
│ OAuth Proxy: /api/notify endpoint │
│ │
│ { │
│ notificationType: "alert", │
│ title: "Alfred", │
│ message: "Check the laundry", │
│ priority: "default", │
│ sound: true, │
│ vibrate: true │
│ } │
│ │
│ 1. Create notification event │
│ 2. Broadcast to all connected WebSocket clients │
│ connectedClients.forEach(client => { │
│ client.send(JSON.stringify({ │
│ type: "event", │
│ event: "mobile.notification", │
│ payload: {...} │
│ })) │
│ }) │
└──────┬─────────────────────────────────────────────────────┘
│ WebSocket broadcast
┌────────────────────────────────────────────────────────────┐
│ All Connected Mobile Clients │
│ │
│ GatewayClient.handleEvent() │
│ event: "mobile.notification" │
│ → GatewayListener.onNotification() │
│ │
│ MainScreen.onNotification() │
│ 1. Add icon based on type (⏰ ⚠️ 🔔) │
│ 2. Show system notification (background OR timer/remind) │
│ NotificationHelper.showNotification() │
│ 3. Add to chat if foreground │
│ 4. Optional TTS if enabled │
└────────────────────────────────────────────────────────────┘
```
## Token Storage Locations
### Mobile App (Android)
```
File: /data/data/com.openclaw.alfred/shared_prefs/auth_prefs.xml
<map>
<string name="access_token">eyJhbGciOiJSUzI1NiIs...</string>
<long name="token_expiry">1707089234567</long>
<string name="refresh_token">YOUR_REFRESH_TOKEN...</string>
</map>
Access: AuthManager.kt
- getAccessToken()
- isTokenExpired()
- refreshToken()
```
### OAuth Proxy (Memory)
```
Location: In-memory Set (JavaScript)
const connectedClients = new Set();
Each WebSocket connection:
- OAuth token validated once on connect
- Connection stored in Set for broadcasts
- Removed from Set on disconnect
```
### OpenClaw Gateway
```
Location: ~/.openclaw/config.json
{
"gateway": {
"token": "YOUR_OPENCLAW_TOKEN",
"port": 18789,
"bind": "loopback"
}
}
Used by: OAuth Proxy only (localhost connection)
```
### Authentik (Database)
```
Location: PostgreSQL database (10.0.1.75)
Tables:
- oauth2_provider_accesstoken
- oauth2_provider_refreshtoken
- oauth2_provider_authorizationcode
Validates tokens via userinfo endpoint
```
## Network Topology
```
Internet
└─── Router/Firewall (10.0.1.1)
├─── HAProxy VM (10.0.1.20:443)
│ - SSL termination
│ - Reverse proxy
├─── Authentik (10.0.1.75:443)
│ - OAuth2 provider
│ - User authentication
└─── Windows Desktop (10.0.1.100)
- WSL Ubuntu 22.04
├─── OAuth Proxy (port 18790)
│ - LAN-accessible
│ - Validates & proxies
└─── OpenClaw Gateway (port 18789)
- Localhost-only
- Main agent session
```
## Security Model
### Defense in Depth
1. **Mobile App**
- OAuth tokens stored in encrypted SharedPreferences
- Token expiry validation (30s buffer)
- Automatic refresh on expiry
2. **Transport Security**
- TLS 1.3 encryption (mobile → HAProxy)
- Valid SSL certificate
- HTTPS/WSS only for external connections
3. **HAProxy**
- SSL termination
- Rate limiting (if configured)
- IP filtering (if configured)
4. **OAuth Proxy**
- Validates OAuth token on every connection
- No token caching (validates with Authentik)
- Localhost-only connection to OpenClaw
- Injects OpenClaw token (never exposed to client)
5. **OpenClaw Gateway**
- Bind: loopback (127.0.0.1 only)
- Token authentication required
- Not directly accessible from network
6. **Authentik**
- OAuth2 standard compliance
- Secure token generation
- User session management
### Token Security
**OAuth Token (Mobile ↔ Proxy)**
- Short-lived (1 hour)
- Refresh token rotation
- Validated on every connection
- Stored securely on device
**OpenClaw Token (Proxy ↔ Gateway)**
- Static token (configured)
- Never leaves localhost
- Never sent to mobile clients
- Injected by proxy
## File Locations
### Mobile App
```
~/.openclaw/workspace/alfred-mobile/
├── app/src/main/java/com/openclaw/alfred/
│ ├── auth/AuthManager.kt (OAuth handling)
│ ├── gateway/GatewayClient.kt (WebSocket client)
│ ├── ui/screens/MainScreen.kt (UI + notification handler)
│ ├── notifications/NotificationHelper.kt
│ └── storage/ConversationStorage.kt
├── ARCHITECTURE.md (this file)
└── secrets.properties (OAuth client ID)
```
### OAuth Proxy
```
~/.openclaw/workspace/alfred-proxy/
├── server.js (Main proxy logic)
├── .env (Config + secrets)
└── /tmp/alfred-proxy.log (Runtime logs)
```
### OpenClaw
```
~/.openclaw/
├── config.json (Gateway token + config)
└── workspace/skills/mobile-notify/ (Notification tool)
├── mobile-notify (CLI wrapper)
└── scripts/notify.js (Implementation)
```
### System Services
```
/etc/systemd/system/alfred-proxy.service (if using systemd)
/tmp/alfred-proxy.log (proxy logs)
```
## Ports Summary
| Service | Port | Bind | Access | Protocol |
|---------|------|------|--------|----------|
| HAProxy | 443 | 0.0.0.0 | External | HTTPS/WSS |
| Authentik | 443 | 10.0.1.75 | LAN | HTTPS |
| OAuth Proxy | 18790 | 0.0.0.0 | LAN | HTTP/WS |
| OpenClaw Gateway | 18789 | 127.0.0.1 | Localhost | HTTP/WS |
## URLs
| Purpose | URL |
|---------|-----|
| Mobile app connection | `wss://alfred-app.example.com` |
| OAuth authorize | `https://auth.example.com/application/o/authorize/` |
| OAuth token | `https://auth.example.com/application/o/token/` |
| OAuth userinfo | `https://auth.example.com/application/o/userinfo/` |
| OAuth redirect | `alfredmobile://oauth/callback` |
| Proxy health check | `http://10.0.1.100:18790/health` |
| Proxy notify API | `http://10.0.1.100:18790/api/notify` |
## Key Design Decisions
1. **Two-token architecture**
- OAuth token: Mobile app authentication
- OpenClaw token: Backend service authentication
- Separation of concerns + security
2. **Proxy pattern**
- Mobile app never has direct OpenClaw access
- Token injection at proxy layer
- Enables centralized validation
3. **Localhost-only OpenClaw**
- Reduces attack surface
- Proxy is single point of entry
- Gateway not exposed to network
4. **Notification broadcasting**
- All clients receive notifications
- Supports multiple devices
- Real-time push via WebSocket
5. **SSL termination at HAProxy**
- Centralized certificate management
- Backend uses plain HTTP (trusted LAN)
- Standard reverse proxy pattern
## Version
1.0.0 - Initial architecture (February 2026)