Archived
101 lines
4.0 KiB
Markdown
101 lines
4.0 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Project Overview
|
|
|
|
This is a live transcription desktop application built with Electron that provides real-time speech-to-text transcription with OBS integration. The app supports multiple transcription backends including local/remote Whisper and ElevenLabs Scribe, with GPU acceleration support.
|
|
|
|
## Architecture
|
|
|
|
### Main Components
|
|
|
|
- **Electron Main Process** (`src/main.js`): Manages application windows, starts the Express server, handles IPC communication between main window and display window
|
|
- **Express Server** (`src/server/index.js`): REST API server with Socket.IO for real-time communication, handles transcription requests and settings management
|
|
- **Frontend Client** (`public/js/app.js`): Main UI for recording controls, settings, and transcription display
|
|
- **Display Window** (`public/display.html`): Transparent overlay window for OBS capture
|
|
|
|
### Backend Architecture
|
|
|
|
- **Transcription Backends** (`src/server/backends/`):
|
|
- `whisper.js`: Local and remote OpenAI Whisper integration with queue management
|
|
- `elevenlabs.js`: ElevenLabs Scribe API integration
|
|
- **Utilities** (`src/server/utils/`):
|
|
- `settingsManager.js`: Persistent settings storage and management
|
|
- `transcriptionQueue.js`: Queue system for managing audio processing
|
|
- `audioUtils.js`: Audio processing utilities
|
|
|
|
### Key Technologies
|
|
|
|
- **Electron**: Desktop app framework
|
|
- **Express + Socket.IO**: Real-time server communication
|
|
- **MediaRecorder API**: Browser audio recording
|
|
- **Web Audio API**: Audio level monitoring and analysis
|
|
- **Multiple AI APIs**: OpenAI Whisper, ElevenLabs Scribe
|
|
|
|
## Development Commands
|
|
|
|
```bash
|
|
# Start the application in development mode (opens DevTools)
|
|
npm run dev
|
|
|
|
# Start the application normally
|
|
npm start
|
|
|
|
# Build for distribution
|
|
npm run build
|
|
|
|
# Run server only (for testing backend)
|
|
npm run server
|
|
```
|
|
|
|
## Environment Configuration
|
|
|
|
Copy `.env.example` to `.env` and configure:
|
|
- `WHISPER_MODEL`: Local Whisper model size (tiny/base/small/medium/large)
|
|
- `OPENAI_API_KEY`: For remote Whisper API
|
|
- `ELEVENLABS_API_KEY`: For ElevenLabs Scribe
|
|
- `AUDIO_THRESHOLD`: Silence detection sensitivity (0.001-0.1)
|
|
- `SERVER_PORT`: Server port (default: 3000)
|
|
|
|
## Settings System
|
|
|
|
The app uses a persistent settings system (`src/server/utils/settingsManager.js`) that:
|
|
- Stores settings in `~/.live-transcription/settings.json`
|
|
- Provides UI override of environment variables
|
|
- Handles secure API key storage
|
|
- Syncs settings between client and server
|
|
|
|
## Audio Processing Flow
|
|
|
|
1. **Audio Capture**: MediaRecorder captures microphone input in configurable chunks (1-5 seconds)
|
|
2. **Level Monitoring**: Web Audio API provides real-time audio level feedback
|
|
3. **Transcription**: Audio chunks sent to selected backend (Whisper Local/Remote or ElevenLabs)
|
|
4. **Queue Management**: Transcription queue prevents overload and manages processing order
|
|
5. **Display**: Results broadcast via Socket.IO to main window and OBS display window
|
|
|
|
## Display Window System
|
|
|
|
The app creates two windows:
|
|
- **Main Window**: Controls, settings, and transcription preview
|
|
- **Display Window**: Transparent, always-on-top overlay for OBS capture
|
|
- IPC communication synchronizes settings and transcription data between windows
|
|
|
|
## GPU Acceleration
|
|
|
|
Supports hardware acceleration detection and configuration:
|
|
- **NVIDIA CUDA**: Detected via `nvidia-smi`
|
|
- **AMD ROCm**: Detected via `rocm-smi` (Linux only)
|
|
- **Intel OpenVINO**: Detected via `lspci` for Intel GPUs
|
|
|
|
## Common Development Patterns
|
|
|
|
- Settings are managed centrally and applied to both UI and backend
|
|
- Audio processing uses queues to prevent overwhelming backends
|
|
- Real-time communication via Socket.IO for transcription updates
|
|
- IPC handles communication between Electron windows
|
|
- Error handling includes graceful fallbacks for missing dependencies
|
|
|
|
## Server Deployment
|
|
|
|
The git directory on the servers are /root/whp and once you do a git pull, to sync web files you would run the rsync command for web-files/ to /docker/whp/web/ |