07Real-Time Retrieval · Source Verification

Conversational AI Search Engine

QLoRA-fine-tuned Llama 3.1 with hybrid dense + sparse retrieval and automated citation and fact-verification.

−40%
misinformation
150ms
P95 time-to-first-token
1K+
concurrent users
+35%
NPS

In short

  • −40% misinformation on TruthfulQA-adapted benchmarks; provenance tracking with confidence scoring and claim decomposition (+35% NPS).
  • vLLM streaming serving 1K+ concurrent users on 4×A10G GPUs.
01

Retrieval

Hybrid dense plus sparse retrieval over Qdrant and Elasticsearch, with Kafka feeding real-time updates, behind a QLoRA-fine-tuned Llama 3.1.

02

Verification

  • Automated citation and fact-verification cut misinformation 40% on TruthfulQA-adapted benchmarks.
  • Provenance tracking with confidence scoring and claim decomposition, which moved NPS +35%.
03

Serving

vLLM streaming at 150ms P95 time-to-first-token, holding 1K+ concurrent users on 4×A10G GPUs.