redskyluan

6 ms·

31.
by redskyluan 2y ago
Try ollama? It support several embedding models
32.
Why Manual Sharding Is a Bad Idea for Vector Database and How to Fix It(milvus.io)
8 points by redskyluan 2y ago|3 comments
33.
by redskyluan 2y ago
Sharding can hinder your business growth, especially for all AI startups. Prepare for scalability with a non-sharded architecture from the start.
34.
by redskyluan 2y ago
How is pg performed on windows? Never tried before
35.
by redskyluan 2y ago
Running everything on windows plaform is not easy... maybe just run with a container?
36.
by redskyluan 2y ago
hah... we just try to name it after a bird. "Woodpecker" is a good metaphorical name for a Write-Ahead Log (WAL) service for several reasons of Persistent and Reliable. Woodpeckers repeatedly and persistently peck trees, symbolizi
37.
Woodpecker: A Cloud-Native, Zero-Disk WAL Storage System(github.com)
3 points by redskyluan 2y ago|3 comments
38.
by redskyluan 2y ago
Woodpecker is a modern, Kubernetes-native WAL storage implementation designed for stateless architectures, offering performance without disk overhead. If you've ever wondered what's behind Milvus’s robust log storage engine, now y
39.
by redskyluan 2y ago
I'm actually very disappointed about the performance of Mongo vector search after I test on it. Any vector database, is better than mongodb performance wise.
40.
by redskyluan 2y ago
Amazing！ Search is not a problem . What to search is! Using reasoning model, it is much easier to split task and focus on what to search
41.
by redskyluan 2y ago
github https://github.com/zilliztech/deep-searcher
42.
Deep Searcher, Open source deep researcher on your private data(milvus.io)
2 points by redskyluan 2y ago|1 comments
43.
by redskyluan 2y ago
We have built a RAG calculator that, given a specific model and PDF format, can estimate the embedding and vector database costs. Additionally, it allows for visualizing the results of chunking. We’d love to hear your feedback on how we can
44.
by redskyluan 2y ago
Even SSD won't be fast enough for most indexes due to the random access nature. I've seen more than 1M iops on a huge nvme disk when use DiskANN index
45.
by redskyluan 2y ago
why not try milvus? you get multiple index types, SIMD based brute force search, IVF, HNSW and DiskANN and you never bother to scale
46.
Elasticsearch Is Dead, Long Live Lexical Search(milvus.io)
4 points by redskyluan 2y ago|0 comments
47.
by redskyluan 2y ago
That's true. Now we can offer the new dedicated Cost-optimized CU，pricing is $0.318/hour ($228.96/month) Which means you can store 20M 768 dimension data in 228$ per month
48.
by redskyluan 2y ago
Maybe check this https://zilliz.com/pricing Yan easily store 1B data into Zilliz Serverless and the cost is incredible cheap
49.
by redskyluan 2y ago
Interesting case study, but I'm skeptical of the broader implications. Pinecone is notoriously expensive compared to other vector database services. A horizontal price comparison reveals several better options in the market. Removing t
50.
by redskyluan 2y ago
there seems to be a big hype to adapt pg into any infra. I love PG but this seems not be right thing.
51.
by redskyluan 2y ago
Based on your use cases. I thought it's not hard to push the window to 32K or even 100k if we change the position embedding
52.
by redskyluan 2y ago
Something need to be fixed. Opensource is not about license, it's about people. OSS companies are fighting against cloud vendors to survice. https://zilliz.com/blog/Redis-tightens-its-license-How-can-a...
53.
by redskyluan 2y ago
What I Observe: Simple RAG is Fading, but Complex RAG Will Persist and Evolve - Involving Query Rewriting, Data Cleaning, Reflection, Vector Search, Graph Search, Rerankers, and More Intelligent Chunking. Large Models Should Not Just Be Kno
54.
by redskyluan 2y ago
I would personally not recommend implementing a Redis protocol on top of SQLite, as I've seen too many failed cases like this. Users may perceive your product as a drop-in Redis replacement, and it may work fine in a PoC, but once it h
55.
by redskyluan 2y ago
finally we see some relationship between Golang and AI? I'm tired about the hype of python and rust. I would really appreciate if we saw more AI project written in golang. for example: https://github.com/tmc/langc
56.
by redskyluan 3y ago
What an article! GResNet seems to generated the most similar result from my human evaluation. Maybe we can combine both model with a hybrid search?
57.
by redskyluan 3y ago
what will be the next step of 1 bit? could it be 0.5 bit? Any kind of quantization is not gonna to last long.
58.
by redskyluan 3y ago
By the way, we are looking for talents of velox, data fusion, ducdb, clickhouse to build the world's fastest vector db milvus(mainly on search side). Contact me if you have this background!
59.
by redskyluan 3y ago
Velox could be competitor of datafusion. It is more focus on execution engine and could be great to integrate to other high performance databases. Database will be split into pieces and rebuild!
60.
by redskyluan 3y ago
Great product and great progress. The first step to build rage is always to evaluate. Except all the current evaluations, cost and perf should also be part of evaluations

More ›