antirez

15 ms·

211.
by antirez 8mo ago
Sciascia, btw, is one of the biggest thinkers and writers of '900. It is not really defined by his mafia-related novels and takes. He used to be friend with Borges, and was regarded as one of the top men in humanistic culture. Disclaim
212.
by antirez 8mo ago
Why I do not believe this shows Anthropic serves folks a worse model: 1. The percentage drop is too low and oscillating, it goes up and down. 2. The baseline of Sonnet 4.5 (the obvious choice for when they have GPU busy for the next trainin
213.
by antirez 8mo ago
Can run with mmap() but it is slower. 4-bit quantized there is a decent ratio between the model size and the RAM, with a fast SSD one could try to see how it works. However when a model is 4-bit quantized there is often the doubt that it is
214.
by antirez 8mo ago
Good AI policies (like this one) can be spotted since the TLDR is "Don't submit shitty code". As such, good AI policies should be replaced by "Contribution policies" that says "Don't submit shitty code&quo
215.
by antirez 8mo ago
Probably more complex project than projects 99% of what software developers with a real job do daily.
216.
by antirez 8mo ago
Exactly, the phenomenon is not exactly new :)
217.
by antirez 8mo ago
LLMs can assist you to write a shitload of useless bloatware, or can assist you to take something existing and complicated, and create something minimal that is almost as good. It's up to you.
218.
by antirez 8mo ago
I believe the process of accumulation of knowledge / fixes / interesting ideas will be still valid, so there will be a tons of small projects doing things that you can replicate and throw away, but the foundational libraries &#x2F
219.
by antirez 8mo ago
I believe we will see a new huge wave of useful open source software. However don't expect the development model to stay the same. I was finally able to resurrect a few projects of mine, and many more will come. One incredible thing wa
220.
by antirez 8mo ago
Thanks for your efforts!
221.
by antirez 8mo ago
Also Redis now includes the new Vector Sets data type that is not part of ValKey, and I'm right now working to a new data type that is not a niche use case as Vector Sets. So I expect them to diverge even more in the near future, and t
222.
by antirez 9mo ago
The time needed for an AI patch written with the prompt "now make it as small as possible, clean, and human coded" is as big as reviewing the patch itself.
223.
by antirez 9mo ago
TLDR don't be an asshole and produce good stuff. But I have the feeling that this is not the right direction for the future. Distrust the process: only trust the results. Moreover this policy is strictly unenforceable because good AI u
224.
by antirez 9mo ago
I believe this is actually the center of the matter. Trump surely does not represents all the Americans, but I believe he is currently acting in a so aggressive way because America has a problem, that is: for years governments of other coun
225.
by antirez 9mo ago
Updates: 1. Now it is much faster, and the Python benchmarks were re-done correctly (the benchmark didn't account for model loading, and did warm-up before starting the actual inference, while the C code was tested exactly in the rever
226.
by antirez 9mo ago
Thanks, super interesting remarks. Funny enough I also went the route of implementing it backward :D Initially I totally skipped the text encoding part. Used Python to generate binary blogs. Also the easy 1 step test with image output corre
227.
by antirez 9mo ago
This is to follow the naming of llama.cpp
228.
by antirez 9mo ago
Good idea, noted in the TODO.
229.
by antirez 9mo ago
There are a lot of shortcomings in the current implementation, making it slow (but in my tree is 2x faster as we speak). For instance activations aren't taken in the GPU, kernels are not fused, flash attention is not used, and many oth
230.
by antirez 9mo ago
I think that this is because the current code does a terrible job at taking the activations in the GPU and fusing the kernels. This is the next thing to fix in this implementation indeed.
231.
by antirez 9mo ago
Absolutely true, but now I'll focus on making it fast and I believe it will be possible to go much faster. I left the agent working in the night with a specification and now I'm going to see the progresses and restart the work.
232.
by antirez 9mo ago
lol I saw that video.
233.
by antirez 9mo ago
I cut the difference in speed by half by taking the activations on the GPU. Time to sleep but will continue tomorrow.
234.
by antirez 9mo ago
It is worth it. There is value in making open frontier models simple to run / integrate.
235.
by antirez 9mo ago
Hello, I always used to play with AI. I wrote this, some time ago, just to make an example: https://github.com/antirez/gguf-tools And I have a YouTube channel mostly about AI (in Italian language) where I regularly pos
236.
by antirez 9mo ago
Why it costed me $80? I pay the monthly subscription, and during that time I use it for multiple things. I spent one day of Claude Max on that so it costed me 2 euros and 60 cents.
237.
by antirez 9mo ago
Very slow currently, I added the benchmarks in the README. To go faster it needs to implement inference faster than the current float32-only kernels.
238.
by antirez 9mo ago
One of the most important thing to do right now to redistribute something to the society, is to use AI to write free software: more free software than ever. If AI will be hard to access in the future, the more software it is released free,
239.
by antirez 9mo ago
The two projects have literally nothing in common. Not a line of code, not the approach, nor the design. Nothing. LLMs are not memorization machines that recall every project in the cut & paste terms you could think of.
240.
by antirez 9mo ago
Yes, Opus could check the image to see if it matched the prompt, but I adviced the model to stop and ask the human for a better check and a description of what the cause of the corrupted image could be. But the fact it could catch obvious r

More ›