4 ms·

Bard is a brain-damaged-but-literate idiot compared to GPT 3, which is still dumber than the typical human. Try GPT 4 for a week. I've found it to be more lik

Bard is a brain-damaged-but-literate idiot compared to GPT 3, which is still dumber than the typical human. Try GPT 4 for a week. I've found it to be more like 50% immediately useful, 25% very impressive, and 25% where it's not wrong but I have to poke it a few times with different prompts to coax out the specific answer I'm looking for. That's better than most humans that I collaborate with at work. Literally half of humans -- in a professional IT setting -- can't understand simplified, clear english in emails. Similarly, in my experience about half can't follow simple A -> B logic. Many are perpetually perplexed that prerequisites need to precede the work, not be a footnote in the post-mortem of the predictable failure. Etc... PS: That last sentence is too hard for several English-native speakers I work with to parse. Seriously. I'm not even exaggerating the tiniest bit. I've had coworkers fail to understand words like "orthogonal" or "vanilla" in a sentence. Vanilla! In my estimation, Chat GPT 4 is already smarter than many people, certainly the bottom 25% of the human population. LLMs are a real existential threat to those people in their current state. A few more years of improvement, and they'll be displacing the bottom 50% in workplaces, easily.

dragonwriter 3y ago
> I’ve had coworkers fail to understand words like “orthogonal” or “vanilla” in a sentence. Vanilla! Presumably, you are referring to the idiomatic use of vanilla, which is probably a less-universal idiom than you think it is (it is of fairly recent origin in wide use, derives from a specific American cultural loading of the literal vanilla flavor) and which, even when the general idiom is understood, can rely on a deeply shared understanding of what is the basic default in the referenced context to actually be understand as to its contextual meaning.
ChatGTP 3y ago
Not to be rude, but are you paid by OpenAI to say this? The amount of comments here telling people to “upgrade to ChatGPT 4” is absolutely unprecedented. I know it might be good, but people will find value in it and upgrade if they see the need to do so?
- jiggawatts 3y ago
  I am in no shape, way, or form affiliated with OpenAI or any other AI company. What I and many others have noticed about the "Are LLMs really smart?" debate is that everyone on the "Nay" side is using 3.5 and everyone on the "Yay" side is using 4.0. The naming and the versioning implies that GPT 4 is somehow slightly better than 3.5, like not even a "full +1" better, just "+0.5" better. (This goes to show how trivial it is to trick "mere" humans and their primitive meat brains.) Similarly, all pre-4 LLMs including not just the older ChatGPT variants, but Bard, Vicuna, etc... are all very clearly and obviously sub-par, making glaring mistakes regularly. Hence, people generalise and assume GPT 4 must be more of the same. For the last few weeks, across many forums, every time someone has said "AIs can't do X" I have put X into ChatGPT 4 and it could do it, with only a very few exceptions. The unfortunate thing is that there is no free trial for GPT 4, and the version on Bing doesn't seem to be quite the same. (It's probably too restricted by a very long system prompt.) So no, people won't form their own opinions, at least not yet, because they can't do so without paying for access.
  - omnicognate 3y ago
    I've been paying for GPT-4 since it came out and have used it extensively. It's clearly an iteration on the same thing and behaves in qualitatively the same way. The differences are just differences of degree. It's not hard to get a feel for the "edges" of an LLM. You just need to come up with a sequence of related tasks of increasing complexity. A good one is to give it a simple program and ask what it outputs. Then progressively add complications to the program until it starts to fail to predict the output. You'll reliably find a point where it transitions from reliably getting it right to frequently getting it wrong, and doing so in a distinctly non-humanlike way that is consistent with the space of possible programs and outputs becoming too large for its approach of predicting tokens instead of forming and mentally "executing" a model of the code to work. The improvement between 3.5 and 4 in this is incremental: the boundary has moved a bit, but it's still there.
    - jiggawatts 3y ago
      Most developers -- let alone humans -- I've met can't run trivial programs in their head successfully, let alone complex ones. I've thrown crazy complicated problems at GPT 4 and had mixed results, but then again, I get mixed results from people too. I've had it explain a multi-page SQL query I couldn't understand myself. I asked it to write doc-comments for spaghetti code that I wrote for a programming competition, and it spat out a comment for every function correctly. One particular function was unintelligible numeric operations on single-letter identifiers, and its true purpose could only be understood through seven levels of indirection! It figured it out. The fact that we're debating the finer points of what it can and can't do is by itself staggering. Imagine if next week you could buy a $20K Tesla bipedal home robot. I guarantee you then people would start arguing that it "can't really cook" because it couldn't cook them a Michelin star quality meal with nothing but stale ingredients, one pot, and a broken spatula.
jltsiren 3y ago
> Literally half of humans -- in a professional IT setting -- can't understand simplified, clear english in emails. Similarly, in my experience about half can't follow simple A -> B logic. There are alternate hypotheses. People have preferences. When it appears that someone does not understand something, they may be pretending they don't understand it, or they may simply be ignoring it. Maybe they are trying to avoid an unpleasant task, or maybe they find dealing with a specific person unpleasant and not worth the effort. In my experience, people are far more capable and competent when they feel comfortable and are interested in the task.
- jiggawatts 3y ago
  > In my experience, people are far more capable and competent when they feel comfortable and are interested in the task. That's definitely true, but in my experience people have limits: simple biological ones. Repetitive tasks make practically all humans bored, for example. The fact that AIs never get sleepy, distracted, or bored already makes them super-human in at least that one aspect. That they have essentially perfect English comprehension, and hence aren't phased by the use of jargon or technical language, puts them head-and-shoulders above most humans. The frustrations I'm venting aren't some rare thing. I'm working on a technical team where the project manager doesn't understand what the team members are saying. This is not just a matter of syntax, or jargon. They just don't understand the concepts. This is so common in the wider industry that I'm pleasantly surprised, shocked even, when I come across a PM that can ask useful questions instead of needing endless corrections along the lines of: "It's spelled SQL, not Sequel." I've never met a PM that could do simple arithmetic, like "10 TB at 100 MB/s will take over a day to copy, we should plan for that!". Never. I've tested Chat GPT 4 on both language and concepts that I've seen trip up PMs, and it understood "well enough" every time. For example, GPT 4: The sentence "We deployed sequel server successfully last night" seems incorrect due to the incorrect naming of a product. "Sequel server" should actually be "SQL Server", a popular relational database management system (RDBMS) developed by Microsoft. Therefore, the corrected sentence should be: "We deployed SQL Server successfully last night." PS: If you tell GPT 4 to pretend it is a technical project manager and instruct it to ask followup questions, it is noticeably better at this than any PM I have worked with in the last few years.
- 3y ago
croes 3y ago
>I've found it to be more like 50% immediately useful, 25% very impressive, and 25% where it's not wrong but I have to poke it a few times with different prompts to coax out the specific answer I'm looking for. That could tell us more about your questions than GPT's capabilities.
Al0neStar 3y ago
[flagged]

dragonwriter 3y ago

> I’ve had coworkers fail to understand words like “orthogonal” or “vanilla” in a sentence. Vanilla! Presumably, you are referring to the idiomatic use of vanilla, which is probably a less-universal idiom than you think it is (it is of fairly recent origin in wide use, derives from a specific American cultural loading of the literal vanilla flavor) and which, even when the general idiom is understood, can rely on a deeply shared understanding of what is the basic default in the referenced context to actually be understand as to its contextual meaning.

ChatGTP 3y ago

Not to be rude, but are you paid by OpenAI to say this? The amount of comments here telling people to “upgrade to ChatGPT 4” is absolutely unprecedented. I know it might be good, but people will find value in it and upgrade if they see the need to do so?

jiggawatts 3y ago
I am in no shape, way, or form affiliated with OpenAI or any other AI company. What I and many others have noticed about the "Are LLMs really smart?" debate is that everyone on the "Nay" side is using 3.5 and everyone on the "Yay" side is using 4.0. The naming and the versioning implies that GPT 4 is somehow slightly better than 3.5, like not even a "full +1" better, just "+0.5" better. (This goes to show how trivial it is to trick "mere" humans and their primitive meat brains.) Similarly, all pre-4 LLMs including not just the older ChatGPT variants, but Bard, Vicuna, etc... are all very clearly and obviously sub-par, making glaring mistakes regularly. Hence, people generalise and assume GPT 4 must be more of the same. For the last few weeks, across many forums, every time someone has said "AIs can't do X" I have put X into ChatGPT 4 and it could do it, with only a very few exceptions. The unfortunate thing is that there is no free trial for GPT 4, and the version on Bing doesn't seem to be quite the same. (It's probably too restricted by a very long system prompt.) So no, people won't form their own opinions, at least not yet, because they can't do so without paying for access.
- omnicognate 3y ago
  I've been paying for GPT-4 since it came out and have used it extensively. It's clearly an iteration on the same thing and behaves in qualitatively the same way. The differences are just differences of degree. It's not hard to get a feel for the "edges" of an LLM. You just need to come up with a sequence of related tasks of increasing complexity. A good one is to give it a simple program and ask what it outputs. Then progressively add complications to the program until it starts to fail to predict the output. You'll reliably find a point where it transitions from reliably getting it right to frequently getting it wrong, and doing so in a distinctly non-humanlike way that is consistent with the space of possible programs and outputs becoming too large for its approach of predicting tokens instead of forming and mentally "executing" a model of the code to work. The improvement between 3.5 and 4 in this is incremental: the boundary has moved a bit, but it's still there.
  - jiggawatts 3y ago
    Most developers -- let alone humans -- I've met can't run trivial programs in their head successfully, let alone complex ones. I've thrown crazy complicated problems at GPT 4 and had mixed results, but then again, I get mixed results from people too. I've had it explain a multi-page SQL query I couldn't understand myself. I asked it to write doc-comments for spaghetti code that I wrote for a programming competition, and it spat out a comment for every function correctly. One particular function was unintelligible numeric operations on single-letter identifiers, and its true purpose could only be understood through seven levels of indirection! It figured it out. The fact that we're debating the finer points of what it can and can't do is by itself staggering. Imagine if next week you could buy a $20K Tesla bipedal home robot. I guarantee you then people would start arguing that it "can't really cook" because it couldn't cook them a Michelin star quality meal with nothing but stale ingredients, one pot, and a broken spatula.

jltsiren 3y ago

> Literally half of humans -- in a professional IT setting -- can't understand simplified, clear english in emails. Similarly, in my experience about half can't follow simple A -> B logic. There are alternate hypotheses. People have preferences. When it appears that someone does not understand something, they may be pretending they don't understand it, or they may simply be ignoring it. Maybe they are trying to avoid an unpleasant task, or maybe they find dealing with a specific person unpleasant and not worth the effort. In my experience, people are far more capable and competent when they feel comfortable and are interested in the task.

jiggawatts 3y ago
> In my experience, people are far more capable and competent when they feel comfortable and are interested in the task. That's definitely true, but in my experience people have limits: simple biological ones. Repetitive tasks make practically all humans bored, for example. The fact that AIs never get sleepy, distracted, or bored already makes them super-human in at least that one aspect. That they have essentially perfect English comprehension, and hence aren't phased by the use of jargon or technical language, puts them head-and-shoulders above most humans. The frustrations I'm venting aren't some rare thing. I'm working on a technical team where the project manager doesn't understand what the team members are saying. This is not just a matter of syntax, or jargon. They just don't understand the concepts. This is so common in the wider industry that I'm pleasantly surprised, shocked even, when I come across a PM that can ask useful questions instead of needing endless corrections along the lines of: "It's spelled SQL, not Sequel." I've never met a PM that could do simple arithmetic, like "10 TB at 100 MB/s will take over a day to copy, we should plan for that!". Never. I've tested Chat GPT 4 on both language and concepts that I've seen trip up PMs, and it understood "well enough" every time. For example, GPT 4: The sentence "We deployed sequel server successfully last night" seems incorrect due to the incorrect naming of a product. "Sequel server" should actually be "SQL Server", a popular relational database management system (RDBMS) developed by Microsoft. Therefore, the corrected sentence should be: "We deployed SQL Server successfully last night." PS: If you tell GPT 4 to pretend it is a technical project manager and instruct it to ask followup questions, it is noticeably better at this than any PM I have worked with in the last few years.
3y ago

croes 3y ago

>I've found it to be more like 50% immediately useful, 25% very impressive, and 25% where it's not wrong but I have to poke it a few times with different prompts to coax out the specific answer I'm looking for. That could tell us more about your questions than GPT's capabilities.

Al0neStar 3y ago

[flagged]