antirez

31 ms·

601.
by antirez 2y ago
The most important thing is missing from this post: the performance of Jerpint+Claude. It's not a VS game.
602.
by antirez 2y ago
Indeed.
603.
by antirez 2y ago
Just to clarify: even storing ranking, here would likely produce good results, but not as good as storing the probability, since it exploits better the ability of arithmetic coding to store this fractional intervals. But here the fundamenta
604.
by antirez 2y ago
Oh, that makes sense! So they use the probability of the next token itself. Thanks for clarifying. Also clever trick about the multiple potential tokens to represent the same text.
605.
by antirez 2y ago
I supposed it used arithmetic coding with the ranking bacause they have a distribution easy to exploit: zero more likely, one a bit less and so forth. What's your guess? Unfortunately Bellard is as smart as hermetic. We are here guessi
606.
by antirez 2y ago
Yep, makes sense... Something like 20 years ago I experimented with encoder/decoder models for lossy images compression and it worked very well, but it's a completely different domain indeed, where there aren't single local c
607.
by antirez 2y ago
Yep. For lossy what could work even better is an encoder-decoder model, so that it is possible to just save the embedding, and later the embedding will be turned back into the meaning.
608.
by antirez 2y ago
The way this works is awesome. If I understand correctly, it's like that, given (part of) a sentence, the next token really in the sequence will be one predicted by the model among the top scoring ones, so most next tokens can be mappe
609.
by antirez 2y ago
Haha, the question makes sense per se but there are definitely times where you want to see the output in real time. I'm a big fan of printf-debugging :D
610.
by antirez 2y ago
Sure, I redirect when there is to redirect, but sometimes during debugging you want to spam yourself to see what is happening during some stress testing when some code path is full of printfs. Also sometimes I'm just on the redis-cli a
611.
by antirez 2y ago
Lua is great to embed, but for Redis I picked it as a scripting language with many regrets because I don't like the language itself: it looks almost like designed to have some kind of friction compared to what you expect to write in a
612.
by antirez 2y ago
After a quick test this looks incredibly good and fast. I'll use it as a terminal for the next weeks to see how it goes, but I have good feelings. Thank you so much for writing it. EDIT: WOOOW, for me this is going to be a game changer
613.
by antirez 2y ago
So happy to hear that! Thank you :)
614.
by antirez 2y ago
Writing dump1090 in my parent's house :D Waiting for the only plane crossing the center of the Sicily during xmas time haha. In general, hacking here in Campobello di Licata at day, then at night finding all my friends, drinking with t
615.
by antirez 2y ago
Time is one of the dimensions, but I often see (bad) designers to stick with the first idea they have (and if they are not very good, the first idea is likely very bad), then as the idea shows the weaknesses, instead of understanding what i
616.
by antirez 2y ago
For the academia reward system, maybe Dijkstra was right. But if you work in tech, you quickly discover that most complexity you find around is not introduced because it sells well, but because most people are incapable of good design. Then
617.
by antirez 2y ago
Quantization can be applied exactly in the same way in HNSW. I'm using quantization in the implementation of Redis vector sets and it works very well. I have very big issues with the other points as well, but I prefer to reply to these
618.
by antirez 2y ago
A rewrite, too, would take less than years, would have a better end result. Honestly the FB application is not exactly rocket science, and the massive amount of code they have points more on the direction of broken internal process and a t
619.
by antirez 2y ago
That's not the point, I think: you can implement the brain in BASIC, in theory, this does not means that the brain is per-se a BASIC program. I'll provide a more theoretical framework for reasoning about this: if the way to solve
620.
by antirez 2y ago
But the inference itself is orthogonal to the computation the NN is going. Obviously the inference (and training) are algorithms.
621.
by antirez 2y ago
Let's see that Turing machines can approximate the execution of NN :) That's why there are issues related to numerical precision, but the contrary is also true indeed, NNs can discover and use similar techniques used by traditiona
622.
by antirez 2y ago
It's not made of "steps", it's an almost continuous function to its inputs. And a function is not an algorithm: it is not an object made of conditions, jumps, terminations, ... Obviously it has computation capabilities a
623.
by antirez 2y ago
NNs are not algorithms.
624.
by antirez 2y ago
I wrote the first random semantic that came in mind, in the pseudocode in C above. The fact you believe it is not the right one is a proof that in C I can express the exact semantic , and in Python I can't (because everything is hidde
625.
by antirez 2y ago
// Create n empty arrays. DynamicArray* groups[n_groups]; for (int i = 0; i < n_groups; i++) { groups[i] = create_empty_array(); } // For each element in elements[], append it to it
626.
by antirez 2y ago
A bit later, in the article, you'll see that memory patterns in allocating the arrays have a role. A role that was hidden initially. > Again, what would you suggest instead? The answer is inside you. You have only to search for it.
627.
by antirez 2y ago
That's not the problem at hand. Python is good for pseudocode. But not if you want to talk about cache misses because in the pseudocode written with higher level languages a lot of details on how memory is accessed are opaque.
628.
by antirez 2y ago
Using Python pseudo code in this context is hardly understandable.
629.
by antirez 2y ago
Thank you, I'll encourage my colleagues to take everything we can from ValKey, because this is how it works: as the BSD allowed the fork, the BSD allows less liberal licenses to take back. The magic of very liberal licenses is that: th
630.
by antirez 2y ago
Appreciate your kind words, and I'm very happy it was a good experience to work together, it was cool for me too. I hope there will be ways to exchange infos and hack :)

More ›