Vector Databases Evolve: Similarity Search Becomes a Secondary Index

The Shift from Specialist to Generalist

For years, the industry narrative insisted that vector databases represented a new, distinct category of infrastructure. The argument was simple: embedding vectors are unique mathematical objects that require bespoke storage engines optimized for approximate nearest neighbor (ANN) search. If you built a system that prioritized anything else, you were inherently compromising on the performance of similarity retrieval. However, recent architectural pivots, most notably from Turbopuffer in its V3 redesign, suggest the industry is undergoing a fundamental correction. The vector database is not disappearing; rather, it is losing its status as a distinct architectural container and becoming just another index within a broader retrieval system.

The Gravity of Primary Indexes

The original generation of vector-native databases optimized for the novel challenge: retrieving nearest vectors at scale. By making the ANN structure the primary physical organization of the database, these systems created significant technical debt. When the vector index is the primary index, the data must be physically stored according to its location in vector space. This creates an immediate conflict when you need to perform non-vector tasks. If a document has multiple vector representations, the system must either duplicate the document data for each vector to preserve search performance, or accept significant latency penalties. This is storage amplification. Similarly, as documents are updated, the database must move vectors to maintain clustering quality for the ANN index. Because the document content is coupled to that vector's position, the system triggers massive write amplification, complicating real-time updates for dynamic AI applications.

Multimodal Search Demands Integrated Execution

Modern search requirements rarely rely on a single signal. A standard production query today involves semantic similarity, keyword matching, user permissions, price constraints, and freshness filtering. When the database architecture treats ANN as the sole first-class citizen, these other requirements become second-class additions. This limits the query optimizer's ability to choose the most efficient path. If the engine is physically optimized for vectors, it may be forced to perform a vector search first and filter the result set afterward, even when a structured filter would have narrowed the search space to a fraction of the original data in milliseconds.

Re-evaluating the Database Stack

Relational databases are not defined by their use of B-trees. A B-tree is simply an access structure that facilitates efficient lookups, just as an inverted index handles text or a GIST index handles geographic data. If we view ANN search as an access structure rather than the defining property of the system, the architecture becomes much more flexible. In this model, the database remains responsible for transactionality, durability, and query planning, while the ANN index becomes one of several tools in the toolbox. This convergence is visible from both sides of the market. While vector-first startups are moving toward general-purpose search, legacy players like Elasticsearch have been moving toward vector-first capabilities by integrating embeddings into their existing high-performance indexing pipelines. Both paths lead to the same destination: a unified retrieval engine.

The Conflict of Terminology

An interesting irony in this evolution is the confusion surrounding the word vector. In the context of embeddings, vector refers to the mathematical representation of data. However, database engineers also use vectorization to describe SIMD (Single Instruction, Multiple Data) processing, where CPUs perform operations on arrays of data to maximize throughput. When a database is constrained by a vector-first physical layout, it often struggles to implement the second type of vectorization, which is critical for analytical performance. The most capable modern databases must now handle both: they must support efficient embedding retrieval while simultaneously enabling vectorized execution for aggregation, filtering, and calculation. Systems that cannot do both will eventually be sidelined by engines that treat these as complementary performance optimizations.

The Future of Retrieval Infrastructure

The primary value of a database is its ability to find the right data under complex constraints. As AI applications mature, developers are realizing that their problems are rarely limited to finding the nearest neighbor in a vector space. They need systems that can handle the full lifecycle of data, including permissions, schema-aware filtering, and high-frequency updates. The industry is currently moving away from vector-centric infrastructure and toward retrieval-centric infrastructure. The vector database as a standalone product category is likely to vanish, but the techniques developed during the ANN boom will survive as essential features of the next generation of search databases. The ultimate winner will be the platform that makes the underlying storage complexity invisible, allowing the developer to treat the vector index as just another index, much like a index on a column in a relational table.