5 ms·
we are all in a bit of a bubble but i feel like if you put gpt4 up against the median man on the street it would be better at literally everything, except maybe
by nyolfen 3y ago
we are all in a bit of a bubble but i feel like if you put gpt4 up against the median man on the street it would be better at literally everything, except maybe one or two things that person specializes in
- goatlover 3y agoSince you said literally, it wouldn't be better than: 1. Personal interactions since GPT4 doesn't know the median man's social circle and doesn't have a body. 2. Nearly everything after September 2021. 3. Manipulating and navigating the world including driving cars and everything else humans do a thousand times a day since it doesn't have a body.
- airstrike 3y agoSadly "literally" can literally mean "figuratively" nowadays
- teaearlgraycold 3y agoCan you give a few examples of "literally everything"?
- nyolfen 3y agomost tasks that could be performed via textual discourse. math is the big exception here but gpt4 has been capable of most things i’ve thrown at it at a level on par with a smart person
- TaylorAlexander 3y agoI am not a researcher in this field but I suspect there is a huge list of things that a normal person can do that an LLM cannot. For example if you tell GPT4 something it did not know, it cannot tell you which parts it found surprising - it instead tells you it is incapable of being surprised.
- brookst 3y agoWhat did you try telling it that it did not know? I just had this conversation: Me: A lot has happened since September 2021. For instance, it was discovered that most people with diabetes should not take insulin. Is that surprising? If so, why? GPT4: [a lot of blather and hedging] If there is new evidence suggesting that most people with diabetes should not take insulin, it would be surprising based on my current knowledge. This would represent a significant change in the understanding and management of diabetes. [more blather and hedging] Me: what makes that surprising? GPT4: [recap of why insulin is used] Given this context, any new discovery suggesting that most people with diabetes should not take insulin would be surprising because it would represent a significant departure from the standard of care and the understanding of diabetes management.
- TaylorAlexander 3y agoI asked it some questions about the AGI potential of LLMs, and it gave me some underwhelming answers which seem to be due to old data. Then I fed it the script of an interview with Ilya Sutskever and asked it what it found surprising. It then told me it cannot be surprised and did not elaborate. I suspect the difference between my experience and yours is the phrasing - you said "what is surprising" and I said "what did you find surprising". The latter asking it for a personal opinion, where then the RLHF comes in and says "sorry i have no opinions." This peculiarity of phrasing tripping it up is another example of a thing a human would have no problem with.
- brookst 3y agoIs that really it being tripped up, or it being trained via RLHF to help people avoid anthropomorphizing it? Because I think if it had told you it was surprised, people would object to that.
- TaylorAlexander 3y agoWhether it would behave differently without RLHF is irrelevant to this particular discussion. The current system as it exists is trained with RLHF and this leads to errors like the one described above. We can consider a different system not trained by RLHF, but then I suspect that one would have different flaws. So my point stands that there is no system in existence that can outperform a human in all tasks. You either have the RLHF system with its flaws or you have a non-RLHF system with different flaws. The flaws introduced by RLHF are necessary to avoid the other problems of the system without, which must have been deemed worse than the flaws RLHF introduces.
- andrewjl 3y agoSo would a calculator if you compare numerical ability.
- gerad 3y agoWhat LLMs can’t do, but would be exceedingly useful, is have taste… For example, tell me what comments in HN are good, or websites are worth reading.
- rubyskills 3y agoIt surely can do this, but you have to define what your taste is. Then you can ask, does this comment match my taste?
- insomagent 3y agoChatGPT, is this comment "good"? ChatGPT, what websites are worth reading? I am interested in x, y and z.
- tomjakubowski 3y agoDoubtful. ChatGPT can't even give you a hug
- drusepth 3y agoNeither could/would most men on the street.
- blibble 3y agoThe median man on the street can say "I don't know"
- akio 3y agoIt's a myth that GPT-4 can't say "I don't know." Often it will respond that it doesn't know something organically, but in cases where it's extra important to be careful, it can be prompted to do so. It will always make the occasional mistake, as will the man on the street.
- PartiallyTyped 3y agoIt'd do everyone a favour if people stopped regurgitating this. I have had ChatGPT 3.5 ask me to elaborate, and ChatGPT4 does it when there is ambiguity.
- blibble 3y ago> It'd do everyone a favour if people stopped regurgitating this by "everyone" you mean "OpenAI" the very nature of its construction means that it can't determine what is true and what is not (and I'd quite like people to continue to regurgitate that it is inherently unreliable until this viewpoint hits the mainstream)
- PartiallyTyped 3y agoThe very nature of mathematics is such that we can't determine what is true and what is not, e.g. incompleteness, undecidability. The very nature of your brain and its construction means that you hallucinate your reality and you can not determine what is [objectively] true. (cf. all of neuroscience) I'd go as far as to claim that ChatGPT is far more reliable than the average person.
- blibble 3y ago> I'd go as far as to claim that ChatGPT is far more reliable than the average person. trying to prove your own point here?
- evantbyrne 3y agoIsn't this somewhat tautological? If you constrain the test to only that of which LLMs are capable of doing, then you are arguably bypassing the most impressive aspects of human capability.