anuragsingh922/VocRT

VocRT - Personal Realtime Voice-to-Voice AI Solution

https://vocrt.vercel.app

VocRT is a comprehensive, privacy-first Realtime Voice-to-Voice (V2V) solution that enables natural conversations with AI. Built with cutting-edge TTS models, RAG capabilities, and seamless integration, VocRT processes your voice input and responds with high-quality synthesized speech in real-time.

🚀 Key Features

Real-time Voice Processing

• Ultra-low latency voice-to-voice conversion
• High-quality speech synthesis using Kokoro-82M model
• Customizable voice selection with multiple voice options
• Adjustable threshold and silence duration for optimal user experience

Advanced RAG Capabilities

• Multi-format document support: PDF, CSV, TXT, PPT, PPTX, DOC, DOCX, XLS, XLSX
• URL content extraction: Process web pages, Medium blogs, and online PDFs
• Unlimited document uploads without usage limits or billing concerns
• 100% privacy-first approach with local processing

Privacy & Cost Benefits

• No API usage limits or recurring charges
• Complete data privacy - all processing happens locally
• Offline capability use local LLM model if resources allow
• No data sharing with external AI services

🏗️ Architecture Overview

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   React Client  │◄──►│  Express Server │◄──►│  VocRT Engine   │
│   (Frontend)    │    │   (Backend)     │    │   (Python)      │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                                        |
                                                        |
                                 _______________________|
                                │                       │
                                ▼                       ▼
                       ┌──────────────────┐    ┌─────────────────┐
                       │    Embeddings    │    │   Whisper STT   │
                       │   (e5-base-v2)   │    │   Kokoro TTS    │
                       │    Qdrant DB     │    │                 │
                       │   (Vector Store) │    └─────────────────┘
                       └──────────────────┘

📁 Repository Structure

VocRT/
├── backend/         # Express.js server
├── frontend/        # React client application
├── models/          # AI models directory
├── voices/          # Available voice profiles
├── demo/            # Sample audio and demo files
├── .env             # Environment configuration
├── requirements.txt # Python dependencies
└── README.md        # Project documentation

🛠️ Manual Installation

Prerequisites

• Python 3.10 (required)
• Node.js 16+ and npm
• Docker (for Qdrant vector database)
• Git for cloning repositories

Step 1: Clone Repository

git clone https://huggingface.co/anuragsingh922/VocRT
cd VocRT

Step 2: Python Environment Setup

macOS/Linux:

python3.10 -m venv venv
source venv/bin/activate

Windows:

python3.10 -m venv venv
venv\Scripts\activate

Step 3: Install Python Dependencies

pip install -r requirements.txt

If the installation fails (e.g. due to dependency or PyTorch issues), try the following recovery steps:

pip install --upgrade pip setuptools wheel
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cpu
pip install -r requirements.txt

Step 4: Install eSpeak

Ubuntu/Debian:

sudo apt-get update
sudo apt-get install espeak

macOS:

# Install Homebrew if not present
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Install eSpeak
brew install espeak

Windows:

1. Download from eSpeak official website
2. Run installer and follow instructions
3. Add installation path to system PATH environment variable
4. Verify installation: espeak --version

Verification:

espeak "VocRT installation successful!"

Step 5: Backend Setup (Express.js)

cd backend
npm install
npm run dev

Step 6: Frontend Setup (Vite)

cd frontend
npm install
npm run dev

Step 7: Qdrant Vector Database Setup

Documentation: Qdrant Quickstart Guide

# Pull Qdrant image
docker pull qdrant/qdrant

# Start Qdrant container
docker run -p 6333:6333 -p 6334:6334 \
  -v "$(pwd)/qdrant_storage:/qdrant/storage:z" \
  qdrant/qdrant

Access Points:

• REST API: http://localhost:6333
• Web Dashboard: http://localhost:6333/dashboard
• gRPC API: http://localhost:6334

Step 8: Download Required Models

Embedding Model:

Clone e5-base-v2 to models/e5-base-v2

Whisper STT Model:

Choose your preferred Whisper model size:

✅ Just specify the model name(app.py) — it will be automatically downloaded and loaded.

• tiny: Fastest, lower accuracy
• base: Balanced performance
• small: Better accuracy
• medium/large: Highest accuracy, slower processing

Step 9: Environment Configuration

Edit .env file with your API credentials:

# LLM Configuration
OPENAI_API_KEY=your_openai_api_key_here
GEMINI_API_KEY=your_gemini_api_key_here
LLM_PROVIDER=google  # or 'google' for Gemini
LLM_MODEL=gemini-2.0-flash  # or your preferred model

Step 10: Launch VocRT Server

python3 app.py

🎯 Usage Guide

1. Access the application: Navigate to http://localhost:3000
2. Select voice profile: Choose from available voice options
3. Configure settings: Adjust silence duration for optimal performance
4. Add context: Upload documents, provide URLs, or enter text for AI context
5. Start conversation: Begin speaking and enjoy real-time voice responses

📊 Supported Document Formats

🤖 AI Models & Technology Stack

Core Models

• TTS: Kokoro-82M - High-quality text-to-speech
• STT: OpenAI Whisper - Accurate speech recognition
• Embeddings: e5-base-v2 - Semantic text understanding
• LLM: OpenAI GPT / Google Gemini - Natural language processing

Technology Stack

• Backend: Python, Express.js, gRPC
• Frontend: React, Vite
• Database: Qdrant (Vector Database)
• Audio Processing: Whisper, eSpeak, phonemizer

🔧 Performance Optimization

Hardware Recommendations

• CPU: Multi-core processor (4+ cores recommended)
• RAM: 4GB+ for optimal performance
• Storage: SSD for faster model loading
• GPU: Optional, for accelerated inference can reduce latency upto 60%

Configuration Tips

• Modify silence duration for natural conversation flow
• Use smaller Whisper models for faster STT processing
• Enable GPU acceleration if available

🤝 Contributing

We welcome contributions from the community! Here's how you can help:

Ways to Contribute

• 🐛 Bug Reports: Submit issues with detailed reproduction steps
• 💡 Feature Requests: Suggest new capabilities and improvements
• 📝 Documentation: Improve guides, tutorials, and API docs
• 🔧 Code Contributions: Submit pull requests with enhancements

Development Setup

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Commit changes: git commit -m 'Add amazing feature'
4. Push to branch: git push origin feature/amazing-feature
5. Open a Pull Request

📄 License

This project is licensed under the MIT License

🙏 Acknowledgments

Special thanks to the amazing open-source communities:

• Hugging Face - For hosting and maintaining AI models
• Kokoro-82M Team - Exceptional TTS model
• OpenAI Whisper - Revolutionary speech recognition
• Qdrant - High-performance vector database
• React & Node.js communities

📞 Support & Contact

• Email: anuragjadu922@gmail.com

📞 Website

• VocRT: https://vocrt.vercel.app

⭐ If VocRT helps your projects, please consider giving it a star!

Built with ❤️ for the open-source community