Vector Database

What a Vector Database Is and Why It Exists

A vector database is a data system purpose-built to store high-dimensional vectors, known as embeddings, and to retrieve the stored vectors most similar to a query vector. Embeddings are numeric representations produced by machine learning models, where semantic similarity between two items corresponds to geometric closeness between their vectors. A vector database makes that closeness query the primary operation it is optimized for.

The category exists because traditional databases index scalar values for exact-match and range queries, which is a poor fit for similarity over hundreds or thousands of dimensions. Computing exact distances against every stored vector, a brute-force or exhaustive scan, is accurate but scales linearly with collection size and becomes too slow at millions or billions of vectors. Vector databases instead build specialized approximate indexes that trade a small amount of accuracy for large speedups.

The rise of vector databases tracks the rise of embedding models and retrieval-augmented generation (RAG). When applications began grounding large language model outputs in retrieved documents, they needed infrastructure to store embeddings durably and query them at low latency, which a similarity-search library alone does not fully provide.

Core Job: Storing Embeddings and Serving Fast ANN Queries

The central function of a vector database is approximate nearest neighbor (ANN) search: given a query vector, return the k stored vectors with the smallest distance under a chosen metric, without exhaustively comparing against every vector. Common distance and similarity metrics include cosine similarity, Euclidean (L2) distance, and inner product.

ANN is approximate by design. Rather than guaranteeing the true nearest neighbors, an ANN index aims to return most of them while inspecting only a fraction of the dataset. The quality of an approximate result is measured by recall, the fraction of true nearest neighbors actually returned. A practical system tunes its index so that recall is high enough for the application while latency stays within target.

• Recall: fraction of the true nearest neighbors the index returns, the primary accuracy measure for ANN.
• Latency: time to answer a query, traded against recall by tuning how many candidates the index inspects.
• Distance metric: cosine, Euclidean (L2), or inner product, chosen to match how the embedding model was trained.

Index Structures: HNSW, IVF, and Their Tradeoffs

Two index families dominate. HNSW (Hierarchical Navigable Small World) builds a multilayer proximity graph in which each vector links to nearby vectors, and search descends from a sparse top layer to denser lower layers, greedily walking toward the query. The original HNSW method was introduced by Malkov and Yashunin and offers strong recall at low latency, at the cost of higher memory use and slower index build times. Its main search-time tuning knob is ef_search: larger values inspect more candidates, raising recall and latency together.

IVF (Inverted File) takes a partitioning approach. It clusters vectors into a set of cells (often via k-means), and at query time inspects only the cells whose centroids are closest to the query. Its key parameter, nprobe, sets how many cells to scan: more cells means higher recall and higher latency. IVF builds faster and uses less memory than HNSW but generally has a less favorable speed-recall tradeoff. IVF is frequently paired with product quantization (PQ), which compresses vectors into compact codes so that very large collections fit in memory, accepting some additional accuracy loss.

The recurring tradeoff is among recall, latency, and memory. Practical guidance often targets roughly 90 to 95 percent recall, because pushing toward 99 percent can multiply query time while barely improving result quality for semantic search. The right operating point depends on workload, not a universal constant.

• HNSW: graph-based, high recall at low latency, higher memory and slower builds; tuned via ef_search and graph parameters like M.
• IVF: cluster-based, faster builds and lower memory, tuned via nprobe; often combined with product quantization for compression.
• Memory matters: HNSW graphs and uncompressed vectors are memory-heavy, which is why quantization and disk-backed indexes exist.

Beyond Search: Filtering, Hybrid Search, Sharding, and Multi-Tenancy

A vector database adds operational features that a raw ANN library lacks. Metadata filtering lets queries combine vector similarity with structured predicates, for example restricting results to a user, a date range, or a document type. Doing this efficiently alongside ANN is non-trivial, because naive filtering can either degrade recall or force expensive post-filtering.

Hybrid search blends dense vector similarity with sparse lexical signals such as BM25 keyword scoring, which helps when exact terms, names, or codes matter and pure semantic matching misses them. For production scale, vector databases provide sharding to spread vectors across nodes, replication for availability, and multi-tenancy so many isolated datasets share infrastructure safely. They also handle durable persistence, incremental inserts and deletes, and consistency concerns that a static in-memory index does not address.

• Metadata filtering: combine similarity search with structured constraints without crippling recall.
• Hybrid search: fuse dense vector scores with sparse keyword scoring like BM25.
• Operational layer: sharding, replication, multi-tenancy, persistence, and live updates.

Managed vs Self-Hosted, and Dedicated DBs vs Postgres pgvector

Two orthogonal choices shape adoption. The first is managed versus self-hosted. A managed, serverless offering removes capacity planning and operations in exchange for vendor lock-in and usage-based cost. A self-hosted open-source engine gives control and cost predictability but requires running and scaling the infrastructure.

The second choice is a dedicated vector database versus extending an existing relational database. The pgvector extension adds vector columns and ANN indexing to PostgreSQL, supporting both HNSW and IVFFlat index types as well as exact search. When PostgreSQL is already the system of record, pgvector keeps vectors, metadata, and transactional data in one place, simplifying joins, consistency, and operations. A separate dedicated vector database tends to justify itself only when scale, query throughput, or specialized indexing needs exceed what an extension comfortably handles.

• Managed: zero-ops and elastic scaling, at the cost of lock-in and usage-based pricing.
• Self-hosted: control and cost predictability, with operational responsibility.
• pgvector: keeps embeddings beside relational data in PostgreSQL, supporting HNSW and IVFFlat indexes.

Representative Options as of 2026

The vendor landscape shifts quickly, so the following are representative examples as of 2026 rather than an endorsement or a fixed ranking. Verify current capabilities, licensing, and pricing against official documentation before deciding.

Pinecone is a fully managed, serverless vector database emphasizing zero operations and automatic scaling, available only as a hosted cloud service. Weaviate is an open-source engine known for hybrid search and built-in modules that can generate embeddings from raw input. Qdrant is an open-source engine written in Rust and released under the Apache 2.0 license, often cited for strong filtered-search performance. Milvus is an open-source database under the Apache 2.0 license designed for very large, distributed deployments reaching billions of vectors. Chroma is an open-source database favored for developer experience and prototyping. pgvector is the PostgreSQL extension for teams that want vector search inside their existing relational database.

• Pinecone: fully managed, serverless, operations-light, cloud-only (as of 2026).
• Weaviate: open source, hybrid search, optional built-in vectorization (as of 2026).
• Qdrant: open source, Rust, Apache 2.0, filter-heavy workloads (as of 2026).
• Milvus: open source, Apache 2.0, distributed, billion-scale (as of 2026).
• Chroma: open source, developer-experience focused (as of 2026).
• pgvector: PostgreSQL extension, HNSW and IVFFlat indexes (as of 2026).

How a Vector Database Fits a RAG or AI-Memory Architecture

In a RAG pipeline, source content is split into chunks, each chunk is converted to an embedding by an embedding model, and those embeddings are written to the vector database with associated metadata. At query time, the user question is embedded, the vector database returns the most similar chunks, and those chunks are inserted into the language model prompt as grounding context.

The same pattern underpins AI-memory and second-brain software, where a user's notes, documents, and other captured items are embedded and retrieved by meaning when the user asks a question in natural language. MemX, an AI memory app that stores documents, photos, and voice notes and retrieves them by plain-English questions using OCR and semantic search, is one example of this retrieval pattern. The vector database is the retrieval layer; embedding quality, chunking strategy, and metadata design determine how useful its results are.

Selection Criteria: Scale, Cost, Tunability, Recall, and Operational Burden

Choosing a vector database is mostly about matching workload to constraints. Scale dictates whether a single-node solution or a distributed, horizontally scaled engine is required. Cost spans both infrastructure and engineering time, and a managed service trades higher unit cost for lower operational effort.

Tunability matters when the application has strict latency or recall targets, since exposing index parameters like ef_search or nprobe lets teams find their operating point. Recall requirements differ by use case: a recommendation feed tolerates more approximation than a compliance search. Operational burden, including upgrades, monitoring, backups, and scaling, is often the deciding factor, which is why teams already on PostgreSQL frequently start with pgvector before adopting a dedicated system.

• Scale: current and projected vector count and query throughput.
• Cost: infrastructure plus engineering and operations time.
• Tunability and recall: exposed index parameters and the accuracy the use case demands.
• Operational burden: who runs, monitors, scales, and upgrades it.

Limitations and When a Vector DB Is Overkill

A vector database does not improve embedding quality; poor embeddings yield poor retrieval regardless of index. ANN results are approximate, so edge-case queries can miss relevant items, and pure semantic search can underperform on exact identifiers where keyword or hybrid search is needed. Index builds, especially HNSW, consume significant memory and build time, and very large collections may require quantization that further trades away accuracy.

A dedicated vector database is overkill for small collections. At a few thousand to low hundreds of thousands of vectors, an exact brute-force scan, an in-process library such as FAISS, or pgvector inside an existing PostgreSQL instance is often simpler, cheaper, and accurate. Introducing a separate vector database adds a new system to operate, secure, and keep consistent, a cost only justified when scale or specialized features genuinely require it.

• Approximation can miss relevant results; tune recall and consider hybrid search for exact-term queries.
• Quality is bounded by the embedding model, not the database.
• For small or simple datasets, brute force, an in-process library, or pgvector is often enough.