golol

33 ms·

481.
by golol 4y ago
You are mixing up reinforcement learning and supervised learning here...
482.
by golol 4y ago
I believe ChatGPT and and other assistants will create immense value by providing personal tutoring.
483.
by golol 4y ago
LLMs will for the forseeable future become amazing at simple tasks. Everyone is trying to make them do difficult tasks, which I think is not the right goal, at least for now. Being able to do extremely simple tasks together with a general u
484.
by golol 4y ago
How so? ChatGPT understands Chemistry better than me and I attend the family dinner.
485.
by golol 4y ago
Maybe it is a low-bar, that's subjective. But I do mean understanding, not just knowing some facts. You could say that google was the point where we could say "Pick any human. I claim there exists some subjects that Google has mor
486.
by golol 4y ago
I believe Shopping search will be revolutionized with language models just like general search
487.
by golol 4y ago
In fact, it is quite hilarious that in the last 5 years the go-to criticism of state-of-the art AI has switched from "It can only do one specfic thing very well!" to "It can't do any specific thing very well!". We h
488.
by golol 4y ago
It is good that someone formulated this argument, but I have to say that the conclusion should be obvious to anyone that is not totally blinded by philosophical beliefs about intelligence and algorithms, as opposed to looking at the real ev
489.
by golol 4y ago
Here I am not talking about computational power in this "absolute" sense but in the sense that a model A is fundamentally worse at doing a task than a model B if model A requires factorizing large prime numbers to solve it while m
490.
by golol 4y ago
Hmm I would love for someone else to give their opinion because I find it very interesting but don't quite understand it yet. The way I see it, at the very beginnig the transformer has a large number of choices for numbers a, b that al
491.
by golol 4y ago
Yes exactly. The issues only arise if one doesn't allow the transformer the freedom to use the output for chain of thought. Furthermore we can promot engineer a desired chain of thought behavior but I am afraid that it would be difficu
492.
by golol 4y ago
This is very surprising to me and I don't completely understand it. Let me ask; Does this crucially rely on deterministic behavior of the transformer? The only way I can imagine what you say is that the transformer can see the past inp
493.
by golol 4y ago
As a sidenote and somewhat related: Suppose we train BackwardsGPT on the same sequences of tokens as GPT but reversed. The question is: What would this model be good at? Because it would be ver bad at most things. But some tasks might be ea
494.
by golol 4y ago
By a transformer I essentially mean a model (probably LLM) that operates on a context-window of tokens by appending a new token and then shifting. If I understand correctly the solution you give would be an example of what I mean by using p
495.
What a transformer can NOT do
14 points by golol 4y ago|16 comments