mungoman2

3 ms·

1.
by mungoman2 7d ago
Ohh this is really cool. So one could pack all of common state, every question, every answer in the same prefill, using attention mask to only let them attend to their logical parent. Then additionally do position encoding for token based o
2.
by mungoman2 8d ago
Yes that seems sensible for isolating questions/answers. > 6. Calculate only 64 possible label scores—not the whole vocabulary. This I don’t understand though, could you expand this please?
3.
by mungoman2 8d ago
This is very interesting! Seems like a promising direction. I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV
4.
by mungoman2 11d ago
Now this is why I come here! Amazing thing, will replicate.
5.
by mungoman2 27d ago
This could be interesting but too vague for me to understand what you mean. Could it be explained more clearly, or link some blog entry?
6.
by mungoman2 2mo ago
Well, was the contract terminated? Saying no will sometimes have consequences, and the counterpart is sometimes unreasonable.
7.
by mungoman2 3mo ago
Similarly curious about this. The intuition I extracted: Let’s say we have 10 requests, where 9 of them take 1 second to complete but one that takes 100 seconds. The average time to complete a request is about 10 seconds, but if you experie
8.
by mungoman2 4mo ago
In this case it could be solved by not letting the LLM consume the transaction message. Effectively the same as preventing user supplied info going to the first argument of printf(). The transaction in question can remain opaque to the LLM
9.
by mungoman2 4mo ago
Curious about this take, how do you mean? I understand the point of distorted facts, but what I’m not sure how things are improved by basically having no trust in any facts?
10.
by mungoman2 4mo ago
This looks very interesting. Possible to get those rates without exotic hardware. But I have to say that the comparison is not really fair. Comparison is done with a 2 B model vs frontier models that are likely 100s of times larger. Also ta
11.
by mungoman2 4mo ago
Wow, this is really fascinating. And it reads like the intro of a sci-fi short.
12.
by mungoman2 5mo ago
This looks very cool, but it's not clear what it means. I wonder if it is captivating simply because it syncs cool graphics to audio, like those Winamp visualization filters in the old days.
13.
by mungoman2 5mo ago
Very interesting! Will definitely try this. Feedback: I think it would be beneficial to clarify that there is a trial at the top of the page. Currently we have download and buy buttons next to each other, which I assumed meant that the soft
14.
by mungoman2 5mo ago
Naively it seems odd that running multiple checks in parallel is faster than just running the autoregressive model multiple times in series. It’s the same amount of compute right? But I think the key is that in the standard autoregressive c
15.
by mungoman2 5mo ago
This is an amazing idea, and congrats on getting so far through it. I personally would be wary about fire. Custom electronics without experience, and then putting (assuming here) high energy density batteries in a soft toy handled by little
16.
by mungoman2 5mo ago
This is very interesting and exciting, but IMHO the comparisons read as a bit disingenuous with the other models at 16 bit weights. The 16 bit releases of the others models are not optimized for size, making it difficult to take the compar
17.
by mungoman2 6mo ago
Instead of anchoring the sun and thus noon at the top it would be interesting to have the sun move around the clock face as the year progresses. Noon then moves around as the year progresses. ”Up” could be said to point towards the center o
18.
by mungoman2 6mo ago
What they're saying is that the error for a vector increases with r, which is true. Trivially, with r=0, the error is 0, regardless of how heavily the direction is quantized. Larger r means larger absolute error in the reconstructed
19.
by mungoman2 6mo ago
This is cool. It makes storage of the KV cache much smaller, making it possible to keep more of it in fast memory. Bandwidth-wise it is worse (more bytes accessed) to generate and do random recall on than the vanilla approach, and significa
20.
by mungoman2 6mo ago
Well, the spec can of course define constraints of how the function is implemented.
21.
by mungoman2 6mo ago
Really good. I’ve struggled with the same thing. > Instead of expecting it to understand my requests, I almost always build tooling first to give us a shared language to discuss the project. This is probably the key. I’ve found this to b
22.
by mungoman2 7mo ago
Tbh shrinking the image is probably the cheapest operation you can do that still lets every pixel influence the result. It’s just the average of all pixels, after suitable color conversion.
23.
by mungoman2 7mo ago
This is a very fun idea. Would also be very interesting to see if one could have a system where talking to an NPC could alter the world. One maybe obvious way would be that asking for rumors will actually creates the scenario that the NPC d
24.
by mungoman2 7mo ago
Not sure what the uptime is meant to signal. People have quite low uptime as well…
25.
by mungoman2 7mo ago
I think you’re implying that it would be useful to have the LLM predict the end of the speaker’s speech, and continue with its reply based on that. If, when the speaker actually stops speaking, there is a match vs predicted, the response ca
26.
by mungoman2 8mo ago
But renewable is already cheaper than fossil fuels. Why don't we see this already?
27.
by mungoman2 8mo ago
IMHO, this is not really about AI, it's about setting boundaries and not overwork yourself.
28.
by mungoman2 8mo ago
Good article. Would have liked to see them create a minimal test case, to conclusively show that the results of math operations are actually incorrect.
29.
by mungoman2 8mo ago
Imo the difference is enormous between social media and forums. Infinite feeds are designed to game you for attention, whereas the forums of yore were there to facilitate discussions. I'm sure some forums would also have liked to game
30.
by mungoman2 9mo ago
> Naturally, trains already have track-based location systems, but they are usually based on a train being within a “moving block”, so their accuracy is down to metres rather than centimetres. If you want to monitor track conditions, the

More ›