fulmicoton

10 ms·

61.
by fulmicoton 5y ago
That's on my to-do list for next week. :)
62.
by fulmicoton 5y ago
Absolutely! I want to try that.. We are especially interested in testing the latency minio could offer.
63.
by fulmicoton 5y ago
It depends on the use case does not it. Shared nothing is the best architecture for e-commerce search for instance. But if you have one query every minutes or so for a 1TB dataset, it feels a bit silly to have a couple of servers dedicated
64.
by fulmicoton 5y ago
Thank you @ykevinator!
65.
by fulmicoton 5y ago
It is a web search engine. As explained in the blog post, we made the demo by generating 18k snippets and pushing them to a NLP pipeline that tries to extract the adjective phrase. The tech below is an inverted index.
66.
by fulmicoton 5y ago
Ah my bad! Yes, Humio (and Loki) are opting for this approach. This does decouple compute and storage in a trivial manner. There is indeed a realm in which this brute force approach is the best approach. We could probably make a 4D chart wi
67.
by fulmicoton 5y ago
That's amazing pricing O_O (drooling)
68.
by fulmicoton 5y ago
I tend to agree :). If we get 1 req/s, even for a dataset of that size, this is not as cost efficient. For that kind of use case, I'd probably start using minio. > Seems comparable to AWS ElasticSearch service costs: > - 3 n
69.
by fulmicoton 5y ago
The poster was referring to the latency of the demo and is absolutely correct. The demo can reach 30s on some query. Half of it is due to fetch 180k document generation, and half of it is a single threaded python code that has nothing to do
70.
by fulmicoton 5y ago
It can work on any object storages. I really want to test on Minio to see performance fly :)
71.
by fulmicoton 5y ago
This demo is not indeed quite misleading. The high response time is due to the fact that we generate 18k snippets to generate the tag cloud. Imagine this is the equivalent of clicking on page 1 to 900 on Google! A "barack obama" p
72.
by fulmicoton 5y ago
This is a complex misunderstanding... First, we are getting better throughput from S3 than I we were using a SATA SSD. (and slower than a NVMe SSD). This is a bit of a secret. Of course, single sequential throughput on S3 sucks. At the end
73.
by fulmicoton 5y ago
The bandwidth is free if you are in the same region.
74.
by fulmicoton 5y ago
We have a storage abstraction that boils down to being able to perform Range queries. Anything that allows us to do range queries is ok. That includes basically all object storage I know of (Google Cloud Storage, Azure Storage, Minio, you n
75.
by fulmicoton 5y ago
There might be much better alternative but it really depends on the nature of your key. Because the crux of S3 is the latency you can also decide to encode the docs in blocks, and retrieve more data than is actually needed. For this demo, t
76.
by fulmicoton 5y ago
We really need to make this clear in our next blog post. This is not grep here. We are using the same datastructure that are used in Elasticsearch or google. We just adapted them to be object storage friendly. I would not call Object Storag
77.
by fulmicoton 5y ago
That sounds interesting indeed. Can you schedule a meeting with me? https://calendly.com/paul-quickwit/30min
78.
Show HN: Search 1B web pages and find out that French people are polite(common-crawl.quickwit.io)
9 points by fulmicoton 5y ago|0 comments
79.
by fulmicoton 6y ago
I'm French too. France is tiny. It is not about the number of bakery you have but how far your are from the closest medium size city. Also, the satellite are not geostationary above the US. They rotate above the Earth and will bring th
80.
by fulmicoton 6y ago
Bridgestone.
81.
by fulmicoton 6y ago
It is using a separate library from the TiDB project.
82.
by fulmicoton 7y ago
tantivy main dev here. Just chiming in to confirm this is an accurate answer.
83.
by fulmicoton 7y ago
Some countries have a tax on wealth in addition to income tax. (Switzerland for instance, maybe US in a few years)
84.
by fulmicoton 7y ago
Is it fast though !?
85.
by fulmicoton 7y ago
No.
86.
by fulmicoton 7y ago
https://twitter.com/minoru_osuka/status/1190236816277790720?...
87.
by fulmicoton 7y ago
I met the author of Bayard a couple of times and had beers with him. Does that count as interactions?
88.
by fulmicoton 7y ago
Precisely yes.
89.
Bitpacking faster when SIMD is not available(fulmicoton.com)
2 points by fulmicoton 7y ago|0 comments
90.
by fulmicoton 7y ago
Can you point us to one search engine you know and we know which uses semantic search?

More ›