antirez

21 ms·

451.
by antirez 1y ago
Maybe because Google is largely responsible, paying for the research, of most of the results we are seeing now. I'm not a Google fan, in the web side, and in their idea of what software engineering is, but they deserve to win the AI ra
452.
by antirez 1y ago
I discovered the data is available up to date. Maybe soon or later I'll repeat and extend the analysis, potentially also using multiple ways to compute the vectors, including SBERT (or better SModernBERT).
453.
by antirez 1y ago
Maybe there isn't enough data for each user for pairs, but I thought about mixing the two approaches (but had no time to do it), that is, to have 350 components like now, for the single word frequency, plus other 350 for the most commo
454.
by antirez 1y ago
I didn't know there was an official API! This explains why the data is so readily available in many sources and formats. That's very cool.
455.
by antirez 1y ago
True, but it's a lot more "soft". They basically say do what you want if you don't violate this list of prohibited uses, which is not open source of course, but the list of prohibited users are, mostly, things already ag
456.
by antirez 1y ago
With Gemma3, Qwen, the new Mistral and DeepSeek, all models stronger than llama of the same size, why one would risk issues?
457.
by antirez 1y ago
It's not "less unique" as the structure of the sentence is what matters: the syntax. But you simply tend to use words with balanced frequency. It's not a bad thing.
458.
by antirez 1y ago
The system uses on purpose those simple words, since they are "tellers" of the style of the user in a context independent way. Burrows papers explain this very well, but in general we want to capture low-level structure, more than
459.
by antirez 1y ago
Are you all from the same town? Another user reported this finding.
460.
by antirez 1y ago
Your observation is really acute: the small difference is due to quantization. When we search for element A, that is int8 quantized by default, the code paths de-quantize it, then re-quantize it and searches. This produces a small loss of p
461.
by antirez 1y ago
That's very interesting as I noticed that certain outliers seemed indeed conscious attempts.
462.
by antirez 1y ago
Probably it used to have when the database was created. Then the comments got removed.
463.
by antirez 1y ago
Indeed, but my problem is: all those vector databases (including Redis!) are always thought as useful in the context of learned embeddings, BERT, Clip, ... But I really wanted to show that vectors are very useful and interesting outside tha
464.
by antirez 1y ago
Imagine the 2D space, it also has the same property! You have three points nearby, and a fourth a bit more distant. 4 best match is 1, but 1 best match is 2 and 3.
465.
by antirez 1y ago
When they are rarely used (a small amount of total words produced), they don't have meaningful statistical info for a match, unfortunately. A few users here reported finding actual duplicated accounts they used in the past.
466.
by antirez 1y ago
Great! Thanks for the ACK.
467.
by antirez 1y ago
Redis supports random projection to a lower dimensionality, but the reality is that projecting a 350d vector into 2d is nice but does not remotely captures the "reality" of what is going on. But still, it is a nice idea to use som
468.
by antirez 1y ago
Thank you, tptacek. I was able to verify, thanks to the Internet Archive caching of "pg" for the post of 3 years ago, that the entries are quite similar in the case of "pg". Consider that it captures just the statistical
469.
by antirez 1y ago
Isn't Gemini 2.5 the proof you don't need social network alike data for training?
470.
by antirez 1y ago
Makes sense, thanks.
471.
by antirez 1y ago
> with 4 bytes added for itself Why one would want to do that? Only reason I can think of is, so that values < 4 can be reserved for future users, like new versions of client-server that agree on 64 bit payload if the first 4 bytes ar
472.
by antirez 1y ago
Solaris is such a masterpiece that resisted any accusation, fortunately.
473.
by antirez 2y ago
I know perfectly I can use the API with any wrapper. I don't do that for choice, my human+AI development style is in the form of the chat, and since I discovered that many models behave differently (especially Gemini 2.5) based on wh
474.
by antirez 2y ago
Gemini 2.5 pro is as powerful as everybody says. I still also use Claude Sonnet 3.7 only because the Gemini web UI has issues... (Imagine creating the best AI and then not allowing to attach Python or C files if not renamed .txt) but the wa
475.
by antirez 2y ago
Yes, HSNWs are not so complex, and they work great. I wrote an implementation myself, it's 2500 lines of code (5x the one of dicroce!), but inside there is binary and int8 quantization and many advanced features (include true deletio
476.
by antirez 2y ago
Llama4 seems in many ways a cut and paste of DeepSeek. Including the shared expert and the high sparsity. It's a DeepSeek that does not work well.
477.
by antirez 2y ago
Missing in the article: "Disclaimer: This article was written before testing Gemini 2.5 PRO"
478.
by antirez 2y ago
The problem is not commas, but the fact the specification does not tell you MUST, in a conforming parser, handle extra commas at the end. While commas are not very useful in this case, they have some value because: 1. They help humans parse
479.
by antirez 2y ago
If you really want to think at it as a probability, think at it as "the probability to express correctly the sentence/idea that was modeled in the activations of the model for that token". Which is totally different than &quo
480.
by antirez 2y ago
LLMs learn from examples where the logits are not probabilities, but how a given sentence continues (only one token is set to 1). So they don't learn probabilities, they learn how to continue the sentence with a given token. We apply s

More ›