Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
hachyderm.io is one of the many independent Mastodon servers you can use to participate in the fediverse.
Hachyderm is a safe space, LGBTQIA+ and BLM, primarily comprised of tech industry professionals world wide. Note that many non-user account types have restrictions - please see our About page.

Administered by:

Server stats:

9K
active users

hachyderm.io: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

Doug Turnbull

Do we think about vector storage completely wrong?

For search, we care about increasing the initial recall set. Which often can be 100s or 1000s.

Then we want to improve the precision over that with a smarter model that can use many things (BM25, numeric features of the data, cos similarity, etc)

Why do we obsess over perfect accuracy in top 10 for vector dbs? Building to those benchmarks have externalities making data structures that are hard to update and manage?

Sep 03, 2023, 11:17 AM·Public·Web
4boosts·8favorites
Public
Vicki Boykis

@softwaredoug This is a great question. In my experience, because the architecture of vector DB search (ie not Elasticsearch) means that initial inference, which is both precision and recall, has to be extremely fast - we don’t have the luxury of having a second model to refine

Public
Doug Turnbull

@vicki exactly!

I suspect people building RAG will (and are) eventually realizing this and realize they can't fine tune embeddings to get at things that are really best captured in tabular features

(or they'll find some hacky awkward way to append them as dimensions to their embeddings)

Public
Vicki Boykis

@softwaredoug I wrote this as a few days ago as its been dawning on me just how much extra work RAG is becoming in what was supposed to be an all-encompassing workflow where the LLM does everything x.com/vboykis/status/169687169

Public
Doug Turnbull

@vicki It's the sound of VC's and cash strapped vendors smelling blood in the water and jumping in :)

Though I'm not sure who are the real sharks and who are the injured minnows right now heh

Public
Vicki Boykis

@softwaredoug I’m sure you’ve seen this one arxiv.org/abs/2308.14963

What struck me most about it is how Elastic is in the game again after sleeping on this stuff for a whole, will definitely create a new interesting market dynamic

arXiv.orgVector Search with OpenAI Embeddings: Lucene Is All You NeedWe provide a reproducible, end-to-end demonstration of vector search with OpenAI embeddings using Lucene on the popular MS MARCO passage ranking test collection. The main goal of our work is to challenge the prevailing narrative that a dedicated vector store is necessary to take advantage of recent advances in deep neural networks as applied to search. Quite the contrary, we show that hierarchical navigable small-world network (HNSW) indexes in Lucene are adequate to provide vector search capabilities in a standard bi-encoder architecture. This suggests that, from a simple cost-benefit analysis, there does not appear to be a compelling reason to introduce a dedicated vector store into a modern "AI stack" for search, since such applications have already received substantial investments in existing, widely deployed infrastructure.
Public
ck

@vicki @softwaredoug they've got a strong install base and friendliness in the enterprise space, even coming from behind it counts for a lot

ExploreLive feeds
About