andy12_

9 ms·

61.
by andy12_ 8mo ago
> Auto-AI translation youtube uses is, bluntly, horrid. Any jokes, even obvious ones, are still fumbled frequently. Youtube auto-translations are horrible indeed, and I say that as someone that has to live with the fact that Youtube deci
62.
by andy12_ 8mo ago
This seems really interesting. While Anthropic tried to use dictionary learning over an existing model to try to extract concepts, this almost feels like training the model alongside the dictionary itself (or rather, the model and the dicti
63.
by andy12_ 8mo ago
You’re not going to believe me when I tell you this, but generating a webpage with HTML is far simpler than generating arbitrary graphics (that look good) with SVGs.
64.
by andy12_ 8mo ago
No. Chain of thought it just the model generating a single answer for longer inside <think></think> tags which are not shown in the final response. The strategy of generating different answers in parallel is something diffe
65.
by andy12_ 8mo ago
Not what he said.
66.
by andy12_ 8mo ago
I'm thinking now that as models get better and better at generating SVGs, there could be a point where we can use them to just make arbitrary UIs and interactive media with raw SVGs in realtime (like flash games).
67.
by andy12_ 8mo ago
No, you can't know that the output of a program is unreliable just from the fact that it outputs one words at a time. I already told you that you can perfectly compile a normal program, like a calculator, into the weights of an autoreg
68.
by andy12_ 8mo ago
Reinforcement learning is not done with random data found on the internet; it's done with curated high-quality labeled datasets. Although there have been approaches that try to apply reinforcement learning to pre-training[1] (to learn
69.
by andy12_ 8mo ago
You do understand that the mechanism through which an auto-regressive transformer works (predicting one token at a time) is completely unrelated to how a model with that architecture behaves or how it's trained, right? You can have bot
70.
by andy12_ 8mo ago
No, your opinion is wrong because the reason some models don't seem to have some "strong opinion" on anything is not related to predicting words based on how similar they are to other sentences in the training data. It's
71.
by andy12_ 8mo ago
Unless the LLM is a base model or just a finetuned base model, it definitely doesn't predict words just based on how likely they are in similar sentences it was trained on. Reinforcement learning is a thing and all models nowadays are
72.
by andy12_ 8mo ago
I think this rather shows that GPT 5.2 Instant, which is the version that he most probably used as a free user, is shit and unsusable for anything.
73.
by andy12_ 8mo ago
LLMs can roleplay taking personal offense, can act and respond accordingly, and that's all that matters. Not every discussion about LLMs capabilities must go down the "they are not sentient" rabbit hole.
74.
by andy12_ 8mo ago
The difference between thinking and no-thinking models can be a little blurry. For example, when doing coding tasks Anthropic models with no-thinking mode tend to use a lot of comments to act as a scratchpad. In contrast, models in thinking
75.
by andy12_ 8mo ago
There is also the slight problem that apparently Opus 4.6 verbalized its awareness of being in some sort of simulation in some evaluations[1], so we can't be quite sure whether Opus is actually misaligned or just good at playing along.
76.
by andy12_ 8mo ago
I'm also sure that some kind of linear architecture is possible. After all, humans don't have N^2 perfect recall either.
77.
by andy12_ 8mo ago
It really isn't sub N^2. The main attention is only O(Nk), but only thanks to a lightning indexer that still has complexity O(N^2). So overall it still has the same complexity; just with a smaller constant factor [1] > DSA reduces t
78.
by andy12_ 8mo ago
I mean it in the sense that tokens that pass some external filter (even if that filter isn't perfect) are from a very different probability distribution than those that an LLM generates indiscriminately. It's a new distribution co
79.
by andy12_ 8mo ago
It doesn't matter that it isn't always correct; some external grounding is good enough to avoid model collapse in practice. Otherwise training coding agents with RL wouldn't work at all.
80.
by andy12_ 8mo ago
No, because in the process they are describing the AIs would only post things they have found to fix their problem (a.k.a, it compiles and passes tests), so the contents posted in that "AI StackOverflow" would be grounded in exter
81.
by andy12_ 8mo ago
How is it not a world model? The latents of the model apparently encode enough information to represent a semi-consistent interactuable world. Seems enough world-modely to me. Besides, we already know that agents can be trained with these w
82.
by andy12_ 8mo ago
Which is a problem that would have been prevented had they not purposefully disabled the ERTMS signaling system to avoid delays.
83.
by andy12_ 8mo ago
In my experience of using it to translate ML work between English->Spanish|Galician, it seems to literally translate jargon too eagerly, to the point that I have to tell it to maintain specific terms in English to avoid it sounding too w
84.
by andy12_ 9mo ago
> how poor the code actually is. Very probably. Apparently, it's literally implemented with a React->Text pipeline and it was so badly implemented that they were having problems with the garbage collector executing too frequently
85.
by andy12_ 9mo ago
> However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations Given how stupidly tedious and error-prone citations are, I have no trou
86.
by andy12_ 9mo ago
The most important context is this image[1] from the Guardia Civil. Using Google Maps, and using as context the tree, post and yellow connection box in the image, we can place its location at 180m before the accident in the tracks of the Ir
87.
by andy12_ 9mo ago
Because AGI is still some years away even if you are optimistic; and OpenAI must avoid going to the ground in the meantime due to lack of revenue. Selling ads and believing that AGI is reachable in the near future is not incompatible.
88.
by andy12_ 9mo ago
> I don't entirely understand your extreme optimism towards LLMs given this proclivity for hallucination Simply because I don't see hallucinations as a permanent problem. I see that models keep improving more and more in this
89.
by andy12_ 9mo ago
The problem is that so far, SOTA generalist models are not excellent at just one particular task. They have a very wide range of tasks they are good at, and good scores in one particular benchmarks correlates very strongly with good scores
90.
by andy12_ 9mo ago
I think the problem is the formulation "If so, AGI can't be far behind". I think that if a model were advanced enough such that it could do Einstein's job, that's it; that's AGI. Would it be ASI? Not necessaril

More ›