Update Packages and CLAUDE.md

This commit is contained in:
2025-08-05 16:30:30 -07:00
parent f4c3a024bd
commit bf92addde9
2 changed files with 118 additions and 11 deletions
+101
View File
@@ -0,0 +1,101 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
This is a live transcription desktop application built with Electron that provides real-time speech-to-text transcription with OBS integration. The app supports multiple transcription backends including local/remote Whisper and ElevenLabs Scribe, with GPU acceleration support.
## Architecture
### Main Components
- **Electron Main Process** (`src/main.js`): Manages application windows, starts the Express server, handles IPC communication between main window and display window
- **Express Server** (`src/server/index.js`): REST API server with Socket.IO for real-time communication, handles transcription requests and settings management
- **Frontend Client** (`public/js/app.js`): Main UI for recording controls, settings, and transcription display
- **Display Window** (`public/display.html`): Transparent overlay window for OBS capture
### Backend Architecture
- **Transcription Backends** (`src/server/backends/`):
- `whisper.js`: Local and remote OpenAI Whisper integration with queue management
- `elevenlabs.js`: ElevenLabs Scribe API integration
- **Utilities** (`src/server/utils/`):
- `settingsManager.js`: Persistent settings storage and management
- `transcriptionQueue.js`: Queue system for managing audio processing
- `audioUtils.js`: Audio processing utilities
### Key Technologies
- **Electron**: Desktop app framework
- **Express + Socket.IO**: Real-time server communication
- **MediaRecorder API**: Browser audio recording
- **Web Audio API**: Audio level monitoring and analysis
- **Multiple AI APIs**: OpenAI Whisper, ElevenLabs Scribe
## Development Commands
```bash
# Start the application in development mode (opens DevTools)
npm run dev
# Start the application normally
npm start
# Build for distribution
npm run build
# Run server only (for testing backend)
npm run server
```
## Environment Configuration
Copy `.env.example` to `.env` and configure:
- `WHISPER_MODEL`: Local Whisper model size (tiny/base/small/medium/large)
- `OPENAI_API_KEY`: For remote Whisper API
- `ELEVENLABS_API_KEY`: For ElevenLabs Scribe
- `AUDIO_THRESHOLD`: Silence detection sensitivity (0.001-0.1)
- `SERVER_PORT`: Server port (default: 3000)
## Settings System
The app uses a persistent settings system (`src/server/utils/settingsManager.js`) that:
- Stores settings in `~/.live-transcription/settings.json`
- Provides UI override of environment variables
- Handles secure API key storage
- Syncs settings between client and server
## Audio Processing Flow
1. **Audio Capture**: MediaRecorder captures microphone input in configurable chunks (1-5 seconds)
2. **Level Monitoring**: Web Audio API provides real-time audio level feedback
3. **Transcription**: Audio chunks sent to selected backend (Whisper Local/Remote or ElevenLabs)
4. **Queue Management**: Transcription queue prevents overload and manages processing order
5. **Display**: Results broadcast via Socket.IO to main window and OBS display window
## Display Window System
The app creates two windows:
- **Main Window**: Controls, settings, and transcription preview
- **Display Window**: Transparent, always-on-top overlay for OBS capture
- IPC communication synchronizes settings and transcription data between windows
## GPU Acceleration
Supports hardware acceleration detection and configuration:
- **NVIDIA CUDA**: Detected via `nvidia-smi`
- **AMD ROCm**: Detected via `rocm-smi` (Linux only)
- **Intel OpenVINO**: Detected via `lspci` for Intel GPUs
## Common Development Patterns
- Settings are managed centrally and applied to both UI and backend
- Audio processing uses queues to prevent overwhelming backends
- Real-time communication via Socket.IO for transcription updates
- IPC handles communication between Electron windows
- Error handling includes graceful fallbacks for missing dependencies
## Server Deployment
The git directory on the servers are /root/whp and once you do a git pull, to sync web files you would run the rsync command for web-files/ to /docker/whp/web/
+17 -11
View File
@@ -13,21 +13,27 @@
"author": "", "author": "",
"license": "MIT", "license": "MIT",
"devDependencies": { "devDependencies": {
"@types/node": "^20.11.0", "@types/node": "^22.0.0",
"electron": "^28.1.0", "electron": "^37.2.5",
"electron-builder": "^24.9.1" "electron-builder": "^26.0.12"
}, },
"dependencies": { "dependencies": {
"axios": "^1.6.5", "axios": "^1.7.0",
"cors": "^2.8.5", "cors": "^2.8.5",
"dotenv": "^16.3.1", "dotenv": "^16.4.0",
"express": "^4.18.2", "express": "^4.21.0",
"form-data": "^4.0.0", "form-data": "^4.0.0",
"multer": "^2.0.0-rc.4", "multer": "^2.0.2",
"node-record-lpcm16": "^1.0.1", "node-record-lpcm16": "^1.0.1",
"openai": "^4.24.1", "openai": "^5.11.0",
"socket.io": "^4.7.4", "socket.io": "^4.8.0",
"socket.io-client": "^4.7.4", "socket.io-client": "^4.8.0",
"ws": "^8.16.0" "ws": "^8.18.0"
},
"overrides": {
"rimraf": "^6.0.1",
"glob": "^11.0.3",
"inflight": "npm:lru-cache@^11.0.0",
"@npmcli/move-file": "npm:@npmcli/fs@^4.0.0"
} }
} }