The Rise and Fall of the Vector DB Category

Joe Christian Bergham discusses the evolution of vector databases, particularly focusing on Pinecone's journey from a pioneer to a company struggling to maintain relevance. Initially thriving by promoting vector databases for AI applications, Pinecone has shifted its messaging to appeal more to developers, reflecting a broader market trend. The conversation highlights increasing competition in the vector database landscape, with new entrants like TurboPuffer offering alternative pricing models. Traditional databases, such as Elasticsearch and Postgres with the PgVector extension, are now incorporating vector search capabilities, leading to a convergence of features that questions the necessity of dedicated vector databases.

The importance of integrating reasoning models with search tools to enhance query formulation for agents and large language models (LLMs) is emphasized. While embeddings are crucial for data representation, effective search requires more than similarity measures, including factors like freshness and authority. The critique extends to the idea that embedding the entire web will yield effective search results, advocating for hybrid queries that utilize metadata.

Embedding-based retrieval's application in large-scale recommender systems is noted, with a convergence of technologies between search systems and recommendation engines. For businesses reliant on search quality, dedicated retrieval engines are preferable, while smaller operations may benefit from using Postgres for vector search alongside other workloads. New users are advised to start with Postgres or MongoDB as transactional and vector stores, stressing the importance of understanding and cleaning data for retrieval-augmented generation (RAG) applications, with the classical BM-25 algorithm serving as a strong baseline.

The shift towards API-based services for easier development is emphasized, alongside skepticism about integrating logic into databases due to differing scaling properties. The complexity of large SQL statements for data transformation and embedding is acknowledged, highlighting the ongoing tension between what should reside in databases versus external systems.

The misconception that RAG is dead due to the perceived decline of vector database infrastructure is addressed, clarifying that RAG remains relevant. The ongoing debate about long context versus RAG suggests that claims of one replacing the other are often misguided. Examples illustrate that RAG may not be necessary for all scenarios, such as with single PDFs, and advancements in context windows make previous approaches less applicable.

In discussing graph databases, their effectiveness in solving specific problems, particularly in traversing graph edges, is noted, along with the challenge of building knowledge graphs. Exploration can also be conducted using search engines, cautioning against the misconception that graph databases are essential for graph-related tasks. The role of knowledge graphs in LLM applications is highlighted, noting that LLMs facilitate the creation of entity triplets, which has historically been a bottleneck.

The conversation shifts to embedding models, mentioning GINA, a European startup, and Voyage, acquired by Nvidia. The need for more domain-specific embedding models in sectors like legal, finance, and health is advocated, along with the use of visual language models to enhance embeddings without complex processing. Challenges in the embedding model market, including the need for API-based solutions and effective cost management, are acknowledged, with hope for the emergence of more general embedding models. Community engagement is encouraged, inviting the audience to connect on Axe, highlighting its unique value for the AI community.