Jensson

5 ms·

31.
by Jensson 2mo ago
If you market your hammer as an all purpose tool, yes. If LLM are worse than humans at predicting data it is valid to wonder why, you could think they would be better at it since they are next token predictors.
32.
by Jensson 2mo ago
Yes, and good senior software engineer is ahead of fable, but benchmarks can't capture that either. We already know from testing humans that test scores don't correlate that well with how effective a person is at work. Same applie
33.
by Jensson 2mo ago
There is high rate of progress in specific domains, not high rate of progress in generalness. The models haven't gotten generally smarter, for things they didn't focus on the models are just as bad as a year ago.
34.
by Jensson 2mo ago
You aren't contradicting the person.
35.
by Jensson 2mo ago
> Do you rely on the surrounding plants and animals in your area for sustenance, creating tools, daily life, etc? Europeans did that a few hundred years ago, almost everyone did all over the world, that is not unique to anywhere.
36.
by Jensson 2mo ago
> They’re saying that a lot of indigenous culture and practice was more sustainable Europe seems very sustainable to me, people have been living there for thousands of years and are still thriving. Why do you say Europeans doesn't h
37.
by Jensson 2mo ago
You know there were lived experiences of black people who didn't mind being slaves as well and were able to roam freely? There is a reason we don't just go by anecdotes, anecdotally white people were very nice to their slaves and
38.
by Jensson 2mo ago
Every population grow to where they barely can support themselves, including Australian natives. It doesn't matter what you do, until we got good contraceptives starvation was a fact in every culture except after periods where food pro
39.
by Jensson 2mo ago
No, the argument is that the best person cannot overestimate his own rank, and the worst person cannot underestimate it. The better you are the less room there is for you to overestimate your skill, second place can at most be off by one et
40.
by Jensson 2mo ago
The graph of random data looks like that since its capped. The first place person cannot overestimate his position, and the bottom place person cannot underestimate his position. So any randomness at all will replicate the effect unless the
41.
by Jensson 2mo ago
Psych 101 says stuff like people being more willing to accept a date with a person after a scary experience since they mistake their quick beating heart for love, rather than humans sees a scary experience as a stronger signal of bonding th
42.
by Jensson 2mo ago
Right, the CoT training step does work a bit more like normal training. But those do produce some very weird results, if you look at the "thoughts" CoT training leads to, I wouldn't say that CoT adds general intelligence, it
43.
by Jensson 2mo ago
Hard in "hard science" doesn't mean hard as in difficulty, it means hard as in not soft. Soft sciences are difficult to explore since they aren't rigid, they move around as you prod at them etc, you can't get a good
44.
by Jensson 2mo ago
My experience with every person was that most of them overestimate what they know regardless of experience level. You just notice that more in new grads since its easier to tell when people are wrong about simple things than when they are w
45.
by Jensson 2mo ago
Humans learn to be a next token predictor as a kid when they learn to speak, an LLM cannot learn to be a next token predictor or anything of the sort, we have no clue how you could have an LLM learn human language just based on a thousands
46.
by Jensson 2mo ago
> transforming an input into an output based on what inputs that part of the brain has previously been exposed to You changed the definition there, for it to be like an LLM it should be: > transforming an input into an output trying t
47.
by Jensson 2mo ago
No, brains doesn't just try to mimic pasts signals, LLMs do that but brains doesn't. The way they make LLM solve problems is by adding a lot of logical jumps into its data, or break down different problems etc, and then as it pred
48.
by Jensson 2mo ago
> I suspect what is much more likely meant, is that token predictors cannot be smart, not now nor in the future after improvements, because they are token predictors and predicting tokens is not how intelligence works. Why do you think t
49.
by Jensson 2mo ago
> Solves what? Chess? No thats not AI, its just a chess bot. Turing test? No, thats not AI, its just a dumb token predictor. You are moving the goalpost here if you think the chess AI was AGI. All those problems were evidence AI wasn&#x2
50.
by Jensson 2mo ago
LLM doesn't just produce output as a function, they are much more specific: they predict text based on text they have been trained on.
51.
by Jensson 2mo ago
> The narrative the brain makes up after the fact for why we did something is not perfectly correlated with the actual reason But it does that introspection, we evolved to make it. If its not useful for anything we wouldn't have evo
52.
by Jensson 2mo ago
You have introspection, you can see a part of your thoughts, you know how that introspective part works since its what we call consciousness, you are conscious about it. That introspection isn't an illusion, what your consciousness see
53.
by Jensson 2mo ago
But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence. So, LLM are just next token predictors, brains are next token predictors +
54.
by Jensson 2mo ago
If it keeps doing dumb things, yeah. But if that actually solves it then those opinions will quickly disappear when it replaces all human white collar work since it does it cheaper and better and faster. AGI is fairly easy to detect for thi
55.
by Jensson 2mo ago
The "dumb" part comes from how it behaves in contexts where it lacks a lot of data, or where the data is skewed. Since they are tuned to give a prediction anyway and just make something up since sometimes those made up things are
56.
by Jensson 2mo ago
"Dumb next token predictor" keeps popping up since that is the core way they work. Since they aren't logic engines but prediction engines they will always return a result regardless what you ask it. Some predictions might be
57.
by Jensson 2mo ago
> It's just that now it looks a lot more polished. Yes, and that is the problem. It used to be if a product looked polished it was fairly polished engineering wise as well if we compare to todays AI slop. You can see that on steam,
58.
by Jensson 2mo ago
The best way to describe the LLM intelligence is "an expert system that works the way people thought expert systems would work". You can encode a massive amount of skills into an LLM, and then the LLM uses those to navigate proble
59.
by Jensson 2mo ago
> Much of an LLM's capability comes from the structure encoded in its learned representations And thats encoded as a set of next token predictions. So the way to see how reliably it solves a problem is to look at the chain of predic
60.
by Jensson 2mo ago
They are useful. They will continue to change the world. They are still next token predictors with all the problems that comes with that. For them to change the world you have to work with them as next token predictors. Ensure that the next

More ›