alfred-mobile/AGENT_TOOLS.md

# Agent Tools - Mobile Notifications & Alarms

**Alfred can send notifications, alerts, and alarms directly to your mobile device using FCM!**

## Overview

The Alfred mobile app supports receiving notifications even when the app is closed, screen is locked, or device is asleep. This enables:

- **Alarms** - High-priority notifications that wake the device
- **Instant alerts** - Get notified when tasks complete
- **Timers** - Set countdown notifications
- **Reminders** - Schedule notifications for specific times
- **Background work** - Alfred can notify you when long-running tasks finish

**Key Features:**
- ✅ **FCM Push Notifications** - Delivery guaranteed even when app is closed
- ✅ **Token Persistence** - Works after proxy restarts
- ✅ **Dual Delivery** - WebSocket (if connected) + FCM (always)
- ✅ **Wake on Alarm** - Device wakes for high-priority alarms

## Features

### ✅ What's Working

1. **Instant Notifications** - Alfred can send alerts immediately
2. **System Notifications** - Show even when app is backgrounded
3. **In-App Display** - Notifications appear in chat when app is open
4. **Multiple Notification Types** - Alert (⚠️), Timer (⏰), Reminder (🔔)
5. **Optional TTS** - Speak notifications when TTS is enabled (foreground only)
6. **WebSocket Broadcast** - All connected mobile devices receive notifications

### 📱 Mobile App Features

- **Background Notifications** - Receive notifications even when app is closed
- **Notification Icons** - Visual indicators for different notification types
- **System Tray** - Notifications appear in Android notification shade
- **Tap to Open** - Tapping notification opens the app
- **Auto-dismiss** - Notifications clear when tapped

## Usage from Alfred

### Send Alarms

**Use `alfred-notify` for all notifications and alarms.**

```bash
# High-priority alarm (wakes device)
alfred-notify --alarm "Wake up!"

# Alarm with custom title
alfred-notify --alarm --title "⏰ Morning Alarm" "Time to get up!"

# Silent alarm (vibrate only)
alfred-notify --alarm --no-sound "Silent wake-up"
```

### Send Notifications

```bash
# Basic notification
alfred-notify "Task completed successfully"

# Notification with custom title
alfred-notify --title "Build System" "Compilation finished"

# Silent notification
alfred-notify --no-sound --no-vibrate "Background update complete"
```

### Schedule via Cron

For scheduled alarms/reminders, use the `cron` tool:

```javascript
// Set alarm for 5 minutes from now
{
  "name": "5 minute alarm",
  "schedule": {
    "kind": "at",
    "atMs": Date.now() + (5 * 60 * 1000)
  },
  "payload": {
    "kind": "agentTurn",
    "message": "Run: alfred-notify --alarm '⏰ 5 minute alarm!'",
    "deliver": false
  },
  "sessionTarget": "isolated"
}
```

**Important:** Always use `sessionTarget: "isolated"` + `agentTurn` for alarms, never `systemEvent`.

## Alfred Integration

Alfred uses `alfred-notify` from cron jobs or via exec tool:

**Example conversation:**
```
You: "Set an alarm for 5 minutes"
Alfred: *schedules cron job* "I'll send an alarm in 5 minutes!"
```

Behind the scenes, Alfred creates a cron job:
```javascript
cron.add({
  schedule: { kind: "at", atMs: Date.now() + 300000 },
  payload: { 
    kind: "agentTurn",
    message: "Run: alfred-notify --alarm '⏰ 5 minute alarm!'"
  },
  sessionTarget: "isolated"
})
```

**Another example:**
```
You: "Let me know when the build finishes"
Alfred: "Sure, I'll notify you when it completes"
```

Alfred monitors the build and executes:
```javascript
exec(['alfred-notify', '--title', 'Build System', 'Build completed! ✅'])
```

## Technical Details

### Architecture

```
┌─────────────────┐
│     Alfred      │ (OpenClaw agent)
│    exec/cron    │
└────────┬────────┘
         │ executes
         ▼
┌─────────────────┐
│ alfred-notify   │ (CLI wrapper)
│   bash script   │
└────────┬────────┘
         │ HTTP POST
         ▼
┌─────────────────┐
│  Alfred Proxy   │ (port 18790)
│   + Firebase    │
│   Admin SDK     │
└────────┬────────┘
         │
         ├──────────────────────┬─────────────────────┐
         │ WebSocket            │ FCM Push            │
         │ (if connected)       │ (always)            │
         ▼                      ▼                     ▼
┌─────────────────┐   ┌─────────────────┐   ┌─────────────────┐
│  Mobile App(s)  │   │ Firebase Cloud  │   │  fcm-tokens     │
│  GatewayClient  │   │   Messaging     │   │  .json (disk)   │
└────────┬────────┘   └────────┬────────┘   └─────────────────┘
         │                     │
         │                     ▼
         │            ┌─────────────────┐
         │            │  Mobile Device  │
         │            │  (even asleep)  │
         │            └────────┬────────┘
         │                     │
         └─────────────────────┘
                      │
                      ▼
            ┌─────────────────┐
            │   Android OS    │ (system notifications)
            └─────────────────┘
```

**Key Features:**
- **Dual Delivery:** WebSocket (instant if connected) + FCM (guaranteed)
- **Token Persistence:** FCM tokens saved to `fcm-tokens.json`, survive restarts
- **Wake Device:** FCM can wake locked/sleeping devices for alarms
- **Firebase Admin SDK:** Uses `cloudmessaging.messages.create` permission

### Event Format

```json
{
  "type": "event",
  "event": "mobile.notification",
  "payload": {
    "notificationType": "alert|timer|reminder",
    "title": "Alfred",
    "message": "Notification text",
    "priority": "default",
    "sound": true,
    "vibrate": true,
    "timestamp": 1706918400000,
    "action": null
  }
}
```

### Mobile App Handling

1. **GatewayClient** receives `mobile.notification` event via WebSocket
2. **MainScreen** `onNotification()` callback processes the event
3. **Notification icons** added based on type (⏰ ⚠️ 🔔 📢)
4. **System notification** shown if timer/reminder OR if app is backgrounded
5. **Chat message** added if app is in foreground
6. **Optional TTS** speaks the notification if enabled (foreground only)

## Testing

### Test Instant Notification

From terminal:
```bash
cd ~/.openclaw/workspace/alfred-proxy
./alfred-notify "Test notification!"
```

### Test Alarm

```bash
./alfred-notify --alarm "Test alarm!"
```

### Test with Custom Title

```bash
./alfred-notify --title "Kitchen Timer" "Oven is ready!"
```

### Test Silent Notification

```bash
./alfred-notify --no-sound --no-vibrate "Silent test"
```

### Test via Alfred

Ask Alfred:
```
"Send me a test alarm"
"Set an alarm for 1 minute"
"Notify me when this finishes"
```

## Monitoring

### Check Proxy Status

```bash
# Service status
systemctl --user status alfred-proxy.service

# Watch logs
tail -f /tmp/alfred-proxy.log

# Check for FCM token registration
grep "fcm.*Registering" /tmp/alfred-proxy.log

# Check for successful sends
grep "fcm.*Successfully" /tmp/alfred-proxy.log
```

### Check Connected Clients & Tokens

When app connects, you'll see:
```
[proxy] New connection from ::ffff:192.168.1.20
[auth] Token validated for user: shadow@dao-mail.com
[fcm] Registering token for user a2b49b91...: ewqRvIsOTuiWJk...
[fcm] Saved tokens to disk
```

### Check Token Persistence

```bash
# View persisted FCM tokens
cat ~/.openclaw/workspace/alfred-proxy/fcm-tokens.json

# Should show:
{
  "user-id-hash": [
    "fcm-token-string"
  ]
}
```

### Test Notification Delivery

```bash
# Via CLI (recommended)
alfred-notify --alarm "Test"

# Via HTTP API
curl -X POST http://localhost:18790/api/notify \
  -H "Content-Type: application/json" \
  -d '{
    "notificationType": "alarm",
    "title": "Test",
    "message": "Hello!",
    "priority": "high",
    "sound": true,
    "vibrate": true
  }'
```

Expected response:
```json
{"success":true,"clients":0,"fcm":1}
```
- `clients`: WebSocket connections (0 if app closed)
- `fcm`: FCM devices notified (should be 1+)

## Troubleshooting

### No notifications appearing

1. **Check FCM tokens persisted**:
   ```bash
   cat alfred-proxy/fcm-tokens.json
   # Should show registered tokens
   ```

2. **Check proxy logs for FCM success**:
   ```bash
   tail -f /tmp/alfred-proxy.log | grep fcm
   # Should show: [fcm] Successfully sent X message(s)
   ```

3. **If seeing "Permission denied" error**:
   ```
   [fcm] Error: Permission 'cloudmessaging.messages.create' denied
   ```
   - Check service account has correct role: **Firebase Admin SDK Administrator Service Agent**
   - Verify FCM API is enabled in Google Cloud Console
   - Regenerate service account key if needed

4. **If "No FCM tokens registered"**:
   - Open Alfred app to trigger token registration
   - Check logs for `[fcm] Registering token`
   - Verify app sends token on connect (should happen automatically)

5. **Test directly**:
   ```bash
   alfred-notify --alarm "Test"
   ```

### WebSocket vs FCM

- **WebSocket**: Instant delivery when app is open and connected
- **FCM**: Always works, even when app is closed or device is asleep
- Both should show in response: `{"clients":1,"fcm":1}`

If `clients:0, fcm:1` → App closed but FCM working (normal)  
If `clients:0, fcm:0` → No tokens registered (open app to fix)

### Cron alarms not firing

1. **Verify cron job uses correct format**:
   ```javascript
   // ✅ CORRECT
   {
     "sessionTarget": "isolated",
     "payload": {
       "kind": "agentTurn",
       "message": "Run: alfred-notify --alarm 'Message'"
     }
   }
   
   // ❌ WRONG
   {
     "sessionTarget": "main",
     "payload": {
       "kind": "systemEvent",
       "text": "Alarm message"
     }
   }
   ```

2. **Check cron job exists**:
   ```bash
   # Ask Alfred in chat
   /cron list
   ```

3. **Check cron execution logs**:
   ```bash
   tail -f /tmp/alfred-proxy.log
   # Look for cron execution and alfred-notify calls
   ```

### Firebase Permission Errors

**Error:** `Permission 'cloudmessaging.messages.create' denied`

**Solution:**
1. Verify service account role: **Firebase Admin SDK Administrator Service Agent** (NOT "Firebase Cloud Messaging Admin")
2. Enable FCM API: https://console.cloud.google.com/apis/library/fcm.googleapis.com
3. Download fresh service account key
4. Place at: `alfred-proxy/service-account.json`
5. Restart proxy: `systemctl --user restart alfred-proxy.service`

See `FCM_SETUP.md` for full setup instructions.

## Future Enhancements

Potential additions:

- [ ] **Custom sounds** - Per-notification sound selection
- [ ] **Action buttons** - "Snooze", "Mark as done", etc.
- [ ] **Deep links** - Open specific app screens from notification
- [ ] **Rich content** - Images, progress bars, etc.
- [ ] **Notification groups** - Collapse similar notifications
- [ ] **Do Not Disturb** - Respect user quiet hours
- [ ] **Multi-device sync** - Mark as read across devices

## Version

1.0.0 - Initial release with alert/timer/reminder support