vecta

Vecta — Vector Search Engine (HNSW · IVF-PQ · 1-bit RaBitQ-style codes · Hybrid BM25+RRF)

Vecta CI Python 3.13 C++20 mypy strict coverage License

Vecta is a from-scratch retrieval engine for RAG and semantic search. It covers the three main approximate-nearest-neighbour families and a hybrid retriever:

The hot loops run in native C++20 kernels that are bit-exact with their NumPy twins: both accumulate in float64, both use (distance, id) ordering, and the C++ build sets -ffp-contract=off. Everything is measured against an exact brute-force oracle with ANN-Benchmarks-style recall@10.

                       Vecta ANN sweep: gauss-64c-96d-20000 (200 queries, k=10)
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ index         ┃ params                              ┃ recall@10 ┃   QPS ┃ bytes/vec ┃ exact dists/q ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ flat          │ exact                               │     1.000 │  1831 │       384 │         20000 │
│ hnsw          │ M=16,ef=16                          │     0.983 │  4826 │       512 │           212 │
│ hnsw          │ M=16,ef=32                          │     0.999 │  1763 │       512 │           281 │
│ hnsw          │ M=16,ef=64                          │     1.000 │  1695 │       512 │           368 │
│ hnsw          │ M=16,ef=128                         │     1.000 │  1146 │       512 │           473 │
│ ivf-pq        │ nlist=141,m=16,nprobe=4,rerank=0    │     0.550 │  2952 │        16 │             0 │
│ ivf-pq        │ nlist=141,m=16,nprobe=16,rerank=0   │     0.550 │   748 │        16 │             0 │
│ ivf-pq        │ nlist=141,m=16,nprobe=8,rerank=50   │     0.968 │  1388 │        16 │            50 │
│ ivf-pq        │ nlist=141,m=16,nprobe=16,rerank=100 │     0.999 │   660 │        16 │           100 │
│ binary+rerank │ bits=96,rerank=50                   │     0.665 │ 14905 │        16 │            50 │
│ binary+rerank │ bits=96,rerank=200                  │     0.951 │  8715 │        16 │           200 │
│ binary+rerank │ bits=96,rerank=500                  │     0.991 │  5090 │        16 │           500 │
└───────────────┴─────────────────────────────────────┴───────────┴───────┴───────────┴───────────────┘
           Native kernels (c++20-native (clang++ -O3)) vs NumPy twins
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━┓
┃ kernel                           ┃ native ms ┃ numpy ms ┃ speedup ┃ bit-exact ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━┩
│ exact L2 top-k (20 queries)      │ 10.81     │ 84.38    │ 7.8x    │ True      │
│ PQ ADC scan (m=16)               │ 0.14      │ 1.53     │ 11.2x   │ True      │
│ Hamming popcount top-k (128-bit) │ 0.06      │ 1.27     │ 20.8x   │ True      │
└──────────────────────────────────┴───────────┴──────────┴─────────┴───────────┘
HNSW graph: 20000 nodes, max level 4, mean L0 degree 13.5, reachable 1.000; filtered (even ids) recall@10 = 1.000

The full report is in benchmarks/results.json, and CI regenerates it on every push.

How to read it:

QPS figures are single-threaded and from a Python driver, so treat them as relative. Exact distances per query is the hardware-independent cost metric.


Architecture

flowchart TB
    subgraph Ingest
        X["float32 embeddings"] --> P["prepare(): contiguous f32, cosine -> unit norm"]
    end
    P --> F["FlatIndex (exact oracle, tombstones)"]
    P --> H["HNSWIndex: levels ~ floor(-ln U / ln M), heuristic links, M / 2M caps"]
    P --> I["IVFPQIndex: k-means++ coarse lists + residual PQ (m x 256 codebooks)"]
    P --> B["BinaryIndex: random orthogonal rotation -> sign bits (dim/8 bytes)"]
    subgraph Native["C++20 kernels (bit-exact NumPy twins)"]
        K1["vecta_l2_topk"]
        K2["vecta_l2_gather"]
        K3["vecta_pq_adc_topk"]
        K4["vecta_hamming_topk (std::popcount)"]
    end
    F --> K1
    H --> K2
    I --> K3
    I -->|rerank| K2
    B --> K4
    B -->|rerank| K2
    D["documents"] --> BM["BM25"]
    H --> RRF["Reciprocal Rank Fusion"]
    BM --> RRF
    RRF --> API["FastAPI collections: /search, /hybrid, metadata filters"]
Module Highlights
csrc/vecta_kernels.cpp Exact L2 top-k with an alive mask, gathered L2 for graph expansion, PQ ADC scan, and Hamming top-k via std::popcount. Every top-k uses a (distance, id) max-heap with -1/+inf padding.
kernels.py Compiles the kernels with a cache keyed on SHA-256 of source and flags, and loads them via ctypes. NumPy twins replicate the C++ accumulation order exactly.
graph.py HNSWIndex: Malkov & Yashunin (2018), including Algorithm 4’s heuristic, which keeps the long-range links between clusters. Filtered and deleted nodes are still traversed but never returned, so filters don’t break connectivity. .npz persistence. Also BinaryIndex.
quantize.py FlatIndex, k-means++ with Lloyd iterations (empty clusters reseeded), ProductQuantizer (Jégou et al., 2011) and IVFPQIndex (residual encoding, nprobe, exact re-ranking).
hybrid.py BM25 with Lucene’s non-negative IDF, and RRF (Cormack et al., 2009).
api.py Named collections (HNSW or flat): add, search with metadata-equality filters, delete, and hybrid search.
bench.py / cli.py Reproducible sweep and kernel benchmark (vecta bench), plus vecta serve.

Quick start

uv sync --python 3.13 --all-extras
uv run vecta bench --output benchmarks/results.json   # ~35 s on a laptop
uv run vecta serve --port 8095                        # http://127.0.0.1:8095/docs
import numpy as np
from vecta import HNSWIndex, IVFPQIndex, clustered_gaussians, recall_at_k, FlatIndex

ds = clustered_gaussians(n=20_000, nq=100, dim=96)
truth = FlatIndex(96); truth.add(ds.base)
gt, _ = truth.search(ds.queries, 10)

hnsw = HNSWIndex(96, m=16, ef_construction=100, metric="l2")
hnsw.add(ds.base)
ids, dists = hnsw.search(ds.queries, k=10, ef=32)
print(recall_at_k(ids, gt, 10))                       # ~0.999
ids, _ = hnsw.search(ds.queries, k=10, filter_fn=lambda i: i % 2 == 0)   # filtered search

ivf = IVFPQIndex(96, nlist=128, m=16).train(ds.base)
ivf.add(ds.base)
ids, _ = ivf.search(ds.queries, k=10, nprobe=16, rerank=100)            # 16 B/vector codes
# Hybrid retrieval over HTTP
curl -X POST localhost:8095/v1/collections -H 'content-type: application/json' \
     -d '{"name":"docs","dim":3,"index":"hnsw","metric":"cosine"}'
curl -X POST localhost:8095/v1/collections/docs/points -H 'content-type: application/json' \
     -d '{"vectors":[[1,0,0],[0,1,0]],"texts":["hnsw graph","bm25 lexical"],"metadata":[{"lang":"en"},{"lang":"de"}]}'
curl -X POST localhost:8095/v1/collections/docs/hybrid -H 'content-type: application/json' \
     -d '{"vector":[0.9,0.1,0],"text":"lexical","k":2}'

Engineering guarantees

Scope and honest limits

References

Malkov & Yashunin, HNSW (TPAMI 2018) · Jégou, Douze & Schmid, Product Quantization for NN Search (TPAMI 2011) · Gao & Long, RaBitQ (SIGMOD 2024) · Arthur & Vassilvitskii, k-means++ (2007) · Robertson & Zaragoza, BM25 (2009) · Cormack, Clarke & Büttcher, RRF (SIGIR 2009) · Aumüller et al., ANN-Benchmarks (2020).


Author & Canonical Portfolio

Engineered by Nagarjuna Reddy Kamireddy (Principal / Lead AI Architect). Released under the Apache-2.0 License.