MonaVec: A Training-Free Embedded Vector Search Kernel for Edge and Offline AI Systems

MonaVec introduces a unique approach to vector search, specifically tailored for environments where traditional server setups and network access are impractical. Unlike conventional systems that rely on large memory capacities or extensive training phases, MonaVec operates similarly to SQLite, functioning with a single file and a straightforward function call. Its innovative quantization mechanism is inherently training-free and does not depend on the input data, utilizing a Randomized Hadamard Transform (RHDH) to standardize input distributions. This allows for efficient quantization to 4 bits without requiring a learned codebook or prior data processing. The results are stored in a compact .mvec file, ensuring reproducibility across different architectures through an embedded ChaCha20 seed, which guarantees consistent outcomes. In tests involving semantic embeddings, MonaVec demonstrated impressive performance, achieving a Recall@10 of 0.960 with a minimal memory footprint, outperforming other leading vector search systems. The implementation, crafted in pure Rust with Python bindings, is optimized for various hardware architectures and is ideal for on-device retrieval and offline applications, mirroring the utility of SQLite for relational data.