What is a vector database and how does it work?

Also called vector store, vector DB, vector search engine

Definition

A vector database is a database built to store embedding vectors and quickly return the vectors most similar to a query vector, usually with approximate nearest neighbor indexes such as HNSW.

A vector database is a database designed to store embeddings and answer similarity queries: given a query vector, return the stored vectors closest to it. Each record pairs a vector with an ID and metadata, such as the source document, a date, or access permissions. Vector databases are the usual retrieval layer in retrieval-augmented generation systems.

How a vector database works#

A vector database works by building an approximate nearest neighbor (ANN) index over the stored vectors. ANN search gives up a small amount of recall in exchange for queries far faster than comparing the query with every stored vector. Common index types:

  • HNSW: a layered proximity graph that search descends from coarse layers to fine ones. Malkov and Yashunin describe logarithmic complexity scaling for HNSW search.
  • IVF: clusters vectors into lists and searches only the lists nearest the query.
  • Quantization: compresses vectors, for example with product quantization, to cut memory use; it is often combined with IVF or HNSW.

A typical query embeds the user's text with the same model used for the documents, applies metadata filters, runs the ANN search with a distance metric (cosine, dot product, or Euclidean), and returns the top k results. Those results often pass to a reranking step. Many vector databases also maintain keyword indexes to support hybrid search.

Vector database vs relational database#

A relational database finds rows by exact matches and ranges on structured columns; a vector database finds records by similarity of meaning.

Relational database Vector database
Query SQL predicates and joins Nearest neighbors of a vector
Match type Exact or range Approximate similarity
Typical index B-tree, hash HNSW, IVF, quantized indexes

The line between the two is blurring. PostgreSQL's pgvector extension adds vector columns with exact search plus HNSW and IVFFlat indexes, so a team can keep vectors next to its relational data. Dedicated vector databases focus on larger indexes, higher query rates, and features such as filtered search and multi-tenancy.

The vector databases directory lists options by type, and What is RAG shows where the database fits in a pipeline.

Cite this definition

The AI Engineering Editors. "What is a vector database and how does it work?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/vector-database

Title
What is a vector database and how does it work?
Author
The AI Engineering Editors
Publisher
The AI Engineering
Updated
View as Markdown

Part of the RAG and vector search hub · Updated