The focus on the top 10 in vector search is a product of wanting to prove value over keyword search. Keyword search is going to miss some conceptual matches. You can try to work around that with tokenization and complex queries with all variations but it's not easy. Vector search isn't all that new a concept. For example, the annoy library (https://github.com/spotify/annoy), an open source embeddings database. - Source: Hacker News / 9 months ago

If you want to go larger you could still use some simple setup in conjunction with faiss, annoy or hnsw. Source: 11 months ago

I then use annoy to compare them. Annoy can use different measures for distance, like cosine, euclidean and more. Source: 12 months ago

Yes you can do this for equality predicates if your row groups are sorted . This blog post (that I didn't write) might add more color. You can't do this for any kind of text searching. If you need to do this with file based storage I'd recommend using a vector based text search and utilize a ANN index library like Annoy. Source: about 1 year ago

If you need large scale (1000+ dimension, millions+ source points, >1000 queries per second) and accept imperfect results / approximate nearest neighbors, then other people have already mentioned some of the best libraries (FAISS, Annoy). Source: about 1 year ago

If you find yourself unsure about the optimization process, leverage the power of benchmarking tools like VectorDBBench. This tool, developed and open-sourced by Zilliz, can evaluate all mainstream vector databases. It allows you to conduct comprehensive experiments and fine-tune your system for optimal performance. - Source: dev.to / 28 days ago

Last week I celebrated my first year at Zilliz 🎉, the startup behind the open source vector database Milvus, in the heart of the AI boom. Somehow, the year has been both the shortest and longest year of my 17 years in the software industry. It seems like a prudent time to stop, catch my breath, and reflect on what I’ve learned. - Source: dev.to / 3 months ago

Zilliz is a powerful vector database designed to empower developers and data scientists in building the next generation of AI and search applications. It offers a robust platform for scalable, efficient, and accurate vector search and analytics, supporting a wide array of AI-driven applications. - Source: dev.to / 4 months ago

As an open-source and self-hosted solution, developers can deploy their own version of the plugin and register it with ChatGPT. The plugin leverages OpenAI embeddings and allows developers to choose a vector database (Milvus, Pinecone, Qdrant, Redis, Weaviate or Zilliz) for indexing and searching documents. Information sources can be synchronized with the database using webhooks. Source: 11 months ago

Zilliz | Onsite/Hybrid (Redwood City, CA, USA) - https://zilliz.com/ Zilliz is the company behind Milvus, the world's most popular open-source vector database. We're hiring engineers, developer advocates, product marketing, and product managers. You can view and apply for open roles at https://zilliz.com/careers, or feel free to reach out to me directly. - Source: Hacker News / about 1 year ago