Multimodal AI.
Local First.
Enterprise Ready.
A generalized offline-first multimodal RAG platform designed for efficient local execution and seamless scalability from personal devices to enterprise environments.
Unified Intelligence Pipeline
Built for the way AI works
Every design decision in OmniWhale-RAG prioritizes local performance, data sovereignty, and production scalability.
Multimodal Intelligence
Unified processing for PDFs, images, charts, and text documents within a single vector space.
Local-First Architecture
Runs fully offline, keeping your data under your complete control — no external API calls required.
Scalable Design
Seamlessly transitions from Milvus Lite on a laptop to a full Milvus Cluster for enterprise workloads.
Hardware Optimization
Leverages Intel Arc 130V GPU and NPU acceleration for efficient on-device AI inference.
Modern Stack
Built on FastAPI, LlamaIndex, Ollama, Milvus, and Next.js — a cohesive, production-grade foundation.
Privacy Focused
All ingestion, embedding, and inference occurs locally. Your data never leaves your machine.
How OmniWhale-RAG works
A clean layered architecture — click any layer to explore what happens inside.
RAG Pipeline
Document Ingestion
PDFs, images, and text are parsed and chunked.
Embedding Generation
Nomic embeddings convert content to dense vectors.
Vector Retrieval
Semantic search finds the most relevant context.
Context Assembly
Retrieved chunks are structured into a prompt.
LLM Generation
Ollama generates a grounded, accurate response.
Modern tools, cohesive foundation
Every dependency is chosen for performance, developer experience, and long-term maintainability.
FastAPI
PythonHigh-performance async Python web framework
LlamaIndex
OrchestrationData framework for LLM-powered RAG applications
Ollama
InferenceLocal LLM runtime with hardware-aware loading
Milvus
Vector DBPurpose-built vector database for embedding search
Nomic Embed
EmbeddingsHigh-quality open-source embedding model
Engineered for on-device AI
OmniWhale-RAG is purpose-built for modern AI-capable hardware, extracting maximum inference performance without a cloud dependency.
Execution Profile
Intel Lunar Lake
Optimized for Intel's Lunar Lake architecture with hardware-specific inference paths.
Arc 130V Graphics
Leverages Intel Arc 130V integrated graphics for parallelized model inference.
NPU Acceleration
Offloads embedding generation to the Neural Processing Unit for sustained throughput.
4-Bit Quantization
Runs quantized models within a 32 GB RAM envelope without sacrificing output quality.
Hardware-Aware Execution
Dynamically selects model loading and inference backends based on available hardware.
One architecture, any scale
OmniWhale-RAG uses a unified codebase across all deployment tiers. Scaling up requires minimal configuration changes — not a rewrite.
Personal Laptop
Milvus LiteSingle-node Milvus Lite with Ollama on a consumer device.
Team Deployment
Local ServerShared FastAPI server with a centralized Milvus instance for small teams.
Department Scale
Milvus ClusterReplicated backend with Milvus Cluster for concurrent multi-user workloads.
Enterprise Scale
EnterpriseDistributed infrastructure with full Milvus Cluster, load balancing, and high availability.
Unified codebase across all tiers
Minimal deployment changes when scaling
Local and distributed support
Up and running in minutes
Five steps from zero to a fully operational local RAG system.
curl -fsSL https://ollama.ai/install.sh | sh
Everything you need to get started
Setup Guide
Complete installation and configuration walkthrough for local and server deployments.
API Documentation
FastAPI endpoint reference with request/response schemas and example queries.
Architecture Overview
Deep-dive into the RAG pipeline, vector indexing strategy, and component boundaries.
GitHub Repository
Browse the full source code, open issues, contribute, or fork for your own deployment.
Ready to run AI locally?
OmniWhale-RAG is open source. Explore the repository, self-host today, or join the waitlist for the managed platform.