Archived
Update Packages and CLAUDE.md
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
This is a live transcription desktop application built with Electron that provides real-time speech-to-text transcription with OBS integration. The app supports multiple transcription backends including local/remote Whisper and ElevenLabs Scribe, with GPU acceleration support.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Main Components
|
||||
|
||||
- **Electron Main Process** (`src/main.js`): Manages application windows, starts the Express server, handles IPC communication between main window and display window
|
||||
- **Express Server** (`src/server/index.js`): REST API server with Socket.IO for real-time communication, handles transcription requests and settings management
|
||||
- **Frontend Client** (`public/js/app.js`): Main UI for recording controls, settings, and transcription display
|
||||
- **Display Window** (`public/display.html`): Transparent overlay window for OBS capture
|
||||
|
||||
### Backend Architecture
|
||||
|
||||
- **Transcription Backends** (`src/server/backends/`):
|
||||
- `whisper.js`: Local and remote OpenAI Whisper integration with queue management
|
||||
- `elevenlabs.js`: ElevenLabs Scribe API integration
|
||||
- **Utilities** (`src/server/utils/`):
|
||||
- `settingsManager.js`: Persistent settings storage and management
|
||||
- `transcriptionQueue.js`: Queue system for managing audio processing
|
||||
- `audioUtils.js`: Audio processing utilities
|
||||
|
||||
### Key Technologies
|
||||
|
||||
- **Electron**: Desktop app framework
|
||||
- **Express + Socket.IO**: Real-time server communication
|
||||
- **MediaRecorder API**: Browser audio recording
|
||||
- **Web Audio API**: Audio level monitoring and analysis
|
||||
- **Multiple AI APIs**: OpenAI Whisper, ElevenLabs Scribe
|
||||
|
||||
## Development Commands
|
||||
|
||||
```bash
|
||||
# Start the application in development mode (opens DevTools)
|
||||
npm run dev
|
||||
|
||||
# Start the application normally
|
||||
npm start
|
||||
|
||||
# Build for distribution
|
||||
npm run build
|
||||
|
||||
# Run server only (for testing backend)
|
||||
npm run server
|
||||
```
|
||||
|
||||
## Environment Configuration
|
||||
|
||||
Copy `.env.example` to `.env` and configure:
|
||||
- `WHISPER_MODEL`: Local Whisper model size (tiny/base/small/medium/large)
|
||||
- `OPENAI_API_KEY`: For remote Whisper API
|
||||
- `ELEVENLABS_API_KEY`: For ElevenLabs Scribe
|
||||
- `AUDIO_THRESHOLD`: Silence detection sensitivity (0.001-0.1)
|
||||
- `SERVER_PORT`: Server port (default: 3000)
|
||||
|
||||
## Settings System
|
||||
|
||||
The app uses a persistent settings system (`src/server/utils/settingsManager.js`) that:
|
||||
- Stores settings in `~/.live-transcription/settings.json`
|
||||
- Provides UI override of environment variables
|
||||
- Handles secure API key storage
|
||||
- Syncs settings between client and server
|
||||
|
||||
## Audio Processing Flow
|
||||
|
||||
1. **Audio Capture**: MediaRecorder captures microphone input in configurable chunks (1-5 seconds)
|
||||
2. **Level Monitoring**: Web Audio API provides real-time audio level feedback
|
||||
3. **Transcription**: Audio chunks sent to selected backend (Whisper Local/Remote or ElevenLabs)
|
||||
4. **Queue Management**: Transcription queue prevents overload and manages processing order
|
||||
5. **Display**: Results broadcast via Socket.IO to main window and OBS display window
|
||||
|
||||
## Display Window System
|
||||
|
||||
The app creates two windows:
|
||||
- **Main Window**: Controls, settings, and transcription preview
|
||||
- **Display Window**: Transparent, always-on-top overlay for OBS capture
|
||||
- IPC communication synchronizes settings and transcription data between windows
|
||||
|
||||
## GPU Acceleration
|
||||
|
||||
Supports hardware acceleration detection and configuration:
|
||||
- **NVIDIA CUDA**: Detected via `nvidia-smi`
|
||||
- **AMD ROCm**: Detected via `rocm-smi` (Linux only)
|
||||
- **Intel OpenVINO**: Detected via `lspci` for Intel GPUs
|
||||
|
||||
## Common Development Patterns
|
||||
|
||||
- Settings are managed centrally and applied to both UI and backend
|
||||
- Audio processing uses queues to prevent overwhelming backends
|
||||
- Real-time communication via Socket.IO for transcription updates
|
||||
- IPC handles communication between Electron windows
|
||||
- Error handling includes graceful fallbacks for missing dependencies
|
||||
|
||||
## Server Deployment
|
||||
|
||||
The git directory on the servers are /root/whp and once you do a git pull, to sync web files you would run the rsync command for web-files/ to /docker/whp/web/
|
||||
+17
-11
@@ -13,21 +13,27 @@
|
||||
"author": "",
|
||||
"license": "MIT",
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.11.0",
|
||||
"electron": "^28.1.0",
|
||||
"electron-builder": "^24.9.1"
|
||||
"@types/node": "^22.0.0",
|
||||
"electron": "^37.2.5",
|
||||
"electron-builder": "^26.0.12"
|
||||
},
|
||||
"dependencies": {
|
||||
"axios": "^1.6.5",
|
||||
"axios": "^1.7.0",
|
||||
"cors": "^2.8.5",
|
||||
"dotenv": "^16.3.1",
|
||||
"express": "^4.18.2",
|
||||
"dotenv": "^16.4.0",
|
||||
"express": "^4.21.0",
|
||||
"form-data": "^4.0.0",
|
||||
"multer": "^2.0.0-rc.4",
|
||||
"multer": "^2.0.2",
|
||||
"node-record-lpcm16": "^1.0.1",
|
||||
"openai": "^4.24.1",
|
||||
"socket.io": "^4.7.4",
|
||||
"socket.io-client": "^4.7.4",
|
||||
"ws": "^8.16.0"
|
||||
"openai": "^5.11.0",
|
||||
"socket.io": "^4.8.0",
|
||||
"socket.io-client": "^4.8.0",
|
||||
"ws": "^8.18.0"
|
||||
},
|
||||
"overrides": {
|
||||
"rimraf": "^6.0.1",
|
||||
"glob": "^11.0.3",
|
||||
"inflight": "npm:lru-cache@^11.0.0",
|
||||
"@npmcli/move-file": "npm:@npmcli/fs@^4.0.0"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user