System Architecture
Understanding the Verdana AI agent architecture with PostgreSQL vector storage, OpenAI embeddings, and web verification powering the EU Green Policies Chatbot
Verdana AI Agent Architecture Overview
My system uses Verdana, an intelligent AI agent that combines query classification, automatic language detection, PostgreSQL vector search, real-time web verification, and OpenAI Whisper speech processing to deliver accurate, context-aware responses about EU Green Deal policies.
PostgreSQL + pgvector
High-performance vector database storing 3072-dimensional OpenAI embeddings of 24+ official EU policy documents for semantic search.
OpenAI Integration
Uses GPT-4 for response generation, text-embedding-3-large for vectorization, and Whisper API for speech-to-text processing.
Verdana AI Agent
Intelligent agent with query classification, language detection, conversation context, and proactive web search capabilities.
Comprehensive Knowledge Base
The system contains 24+ official EU policy documents, processed into 800-token chunks with 300-token overlap, embedded using OpenAI text-embedding-3-large for precise semantic retrieval of relevant policy information.
Official Sources
All documents sourced directly from European Commission official publications and legal databases.
Vector Search
Uses cosine similarity on 3072-dimensional embeddings with 0.3 threshold to find the most relevant policy content for each query.
Semantic Understanding
Advanced semantic search understands context and intent, not just keyword matching.
Verdana Agent Processing Workflow
Verdana handles the complete interaction lifecycle with intelligent query classification, automatic language detection, conversation context awareness, vector search, real-time web verification, and comprehensive source attribution.
1. Query Classification
Verdana classifies queries as casual conversation, identity, or EU Green Deal policy queries and detects user language.
2. Vector Search
PostgreSQL pgvector cosine similarity search using OpenAI embeddings to find relevant EU policy content.
3. Web Verification
Dual web search via Tavily API - EU domain-restricted search plus broader policy research for comprehensive coverage.
4. Context Integration
Verdana integrates conversation history, vector search results, and web verification data for context-aware responses.
5. Source Attribution
Response delivered with comprehensive source attribution, relevance scores, and deduplicated reference list.
Unified Chat Agent Architecture
This streamlined architecture uses a single intelligent agent that combines all necessary capabilities in one efficient system, eliminating the complexity of multiple agent coordination.
Unified Chat Agent
The unified agent consolidates all AI capabilities into a single, efficient system that handles the complete interaction lifecycle. It processes voice input via local Whisper, performs RAG retrieval, conducts web verification, and generates comprehensive responses all within one streamlined workflow.
Speech Processing
- • Local Whisper transcription
- • Multi-language support
- • High-quality audio processing
- • Real-time conversion
RAG Retrieval
- • Semantic document search
- • Vector similarity matching
- • Context-aware retrieval
- • Source ranking & scoring
Web Verification
- • Real-time source checking
- • EU official site priority
- • Policy update detection
- • Accuracy validation
Response Generation
- • Comprehensive synthesis
- • Source attribution
- • Confidence scoring
- • EU AI Act compliance
Architecture Benefits
- • Simplified Architecture: Single agent reduces complexity
- • Faster Processing: No inter-agent communication overhead
- • Better Consistency: Unified decision-making process
- • Enhanced Reliability: Fewer points of failure
- • Easier Maintenance: Single codebase to manage
- • Cost Efficiency: Reduced resource requirements
Technical Stack & Infrastructure
Database Layer
- • PostgreSQL 15+ with pgvector
- • Vector similarity search
- • 1536-dimensional embeddings
- • Efficient indexing & caching
AI & ML Services
- • OpenAI GPT-4o-mini
- • OpenAI Whisper speech-to-text
- • text-embedding-3-large
- • Tavily search API
Backend Services
- • FastAPI with async support
- • LangChain framework
- • WebSocket real-time chat
- • Structured logging
Local Whisper Speech Processing
The system uses OpenAI Whisper deployed locally for high-quality, privacy-focused speech-to-text processing. Audio is processed server-side ensuring consistent quality and eliminating browser compatibility issues.
Whisper Implementation
Frontend Audio Capture
MediaRecorder API captures high-quality audio in multiple formats (WebM, MP4, OGG) for optimal compatibility.
Server-Side Processing
Audio files are sent to backend where Whisper processes them with language detection and confidence scoring.
Docker Integration
Whisper runs in containerized environment with FFmpeg support and optimized resource allocation.
Technical Benefits
Whisper provides state-of-the-art speech recognition quality across multiple languages.
Audio data is processed locally, not sent to external speech recognition services.
Eliminates Web Speech API limitations and browser compatibility issues.
Supported Features
Audio Formats
- • MP3, WAV, WebM
- • M4A, OGG, FLAC
- • Up to 25MB file size
Languages
- • English, French, German
- • Spanish, Italian, Portuguese
- • Dutch, Polish, Romanian
Processing
- • Real-time transcription
- • Automatic language detection
- • Confidence scoring
User Experience & Accessibility
Voice Input
- • OpenAI Whisper integration
- • Multi-language voice recognition
- • Server-side speech-to-text processing
- • High-quality AI transcription
Visual Feedback
- • Recording state indicators
- • Toast notifications
- • Loading animations
- • Error state handling
Session Management
- • Multiple conversation sessions by topic
- • Browser localStorage persistence (private)
- • No external server storage for privacy
- • Auto-session naming and organization
- • Context continuity across sessions
- • Data cleared only with browser cache
Intelligent Verification System
Every user query triggers an automatic verification process that cross-checks RAG results with current web sources, ensuring responses are both comprehensive and up-to-date.
How Verification Works
Step 1: RAG Retrieval
System searches knowledge base for relevant EU policy information using semantic similarity.
Step 2: Web Verification
Simultaneously searches EU official sources for latest updates, policy changes, and current information.
Step 3: Confidence Analysis
Compares RAG and web results, calculates confidence scores, and determines optimal response strategy.
Step 4: Enhanced Response
Generates final response combining verified knowledge base information with current web insights.
Verification Strategies
When web sources confirm RAG information is current and accurate.
When minor updates are detected, blend RAG knowledge with recent web findings.
When significant policy changes are detected, emphasize current web information.