4 ms·

I don't know about GPT-4 but it's very easy to find cases where v3 is incapable of the generalized sort of mathemathical and logic problem solving that humans m

I don't know about GPT-4 but it's very easy to find cases where v3 is incapable of the generalized sort of mathemathical and logic problem solving that humans manage fairly easily, and on that basis alone it didn't classify as an AGI for me. If that can be addressed purely via LLM-technology I'll be impressed (and pretty surprised).

z3c0 4y ago
The big detractor for me is its heavy curation towards predetermined outputs that are constantly being tuned by a small army, essentially making it a Mechanical Turk paired with a nifty model. They can spin it as "trying to improve its usability", but it looks more like "trying to mask its unusability" to me. It's the world's most heavily invested-in parlor trick, and anybody who drops it into a situation of higher stakes will find that out quickly. The GPT-4 paper even warns about this: Most importantly, it still is not fully reliable (it “hallucinates” facts and makes reasoning errors). Great care should be taken when using language model outputs, particularly in high-stakes contexts, with the exact protocol (such as human review, grounding with additional context, or avoiding high-stakes uses altogether) matching the needs of specific applications. Whenever they create a model than can achieve the same outputs without the heavy curation and deliberate bias on certain subjects (like condescending people for asking where to find cigarettes), then I'll be impressed.
pixl97 4y ago
And at the same time it's solving lots of problems that computers have never been able to solve in the past. I'm glad it's not AGI because we'd have much bigger problems, but for a 'language' model it does have interesting methods of rationality, and when put in a verbose mode it seems to do much better. I've not had time to play around and get the interface to see if it can self correct statements it makes like this on first shot. Me: Hello, I am asking you a logic question. Please explain what "it" is in the following sentence Me: "The X doesn't fit in the Y because it is too small" G4: In the sentence "The X doesn't fit in the Y because it is too small," the pronoun "it" refers to "X." The sentence is stating that "X" cannot fit inside "Y" due to its size being too small. Me: "This is incorrect, please give step by step logic on why you came to this conclusion" G4: Apologies for the confusion. I will provide a step-by-step logic for my interpretation: The sentence is: "The X doesn't fit in the Y because it is too small." We need to identify what "it" refers to in the sentence. There are two nouns in the sentence: X and Y. The sentence structure is such that the reason for X not fitting in Y is provided after the word "because." The sentence states that something is "too small," which is the reason for the lack of fit. Since the reason for the lack of fit is being given, we can infer that the pronoun "it" should be referring to one of the two nouns. However, upon closer examination, my previous interpretation was incorrect. Revisiting the analysis, the correct interpretation should be as follows: In the sentence "The X doesn't fit in the Y because it is too small," the pronoun "it" actually refers to "Y." The sentence is stating that "X" cannot fit inside "Y" because "Y" is too small to accommodate "X." My initial interpretation was wrong because I didn't consider that "it" could refer to "Y" and that the sentence is explaining why "X" cannot fit in "Y."

z3c0 4y ago

The big detractor for me is its heavy curation towards predetermined outputs that are constantly being tuned by a small army, essentially making it a Mechanical Turk paired with a nifty model. They can spin it as "trying to improve its usability", but it looks more like "trying to mask its unusability" to me. It's the world's most heavily invested-in parlor trick, and anybody who drops it into a situation of higher stakes will find that out quickly. The GPT-4 paper even warns about this: Most importantly, it still is not fully reliable (it “hallucinates” facts and makes reasoning errors). Great care should be taken when using language model outputs, particularly in high-stakes contexts, with the exact protocol (such as human review, grounding with additional context, or avoiding high-stakes uses altogether) matching the needs of specific applications. Whenever they create a model than can achieve the same outputs without the heavy curation and deliberate bias on certain subjects (like condescending people for asking where to find cigarettes), then I'll be impressed.

pixl97 4y ago

And at the same time it's solving lots of problems that computers have never been able to solve in the past. I'm glad it's not AGI because we'd have much bigger problems, but for a 'language' model it does have interesting methods of rationality, and when put in a verbose mode it seems to do much better. I've not had time to play around and get the interface to see if it can self correct statements it makes like this on first shot. Me: Hello, I am asking you a logic question. Please explain what "it" is in the following sentence Me: "The X doesn't fit in the Y because it is too small" G4: In the sentence "The X doesn't fit in the Y because it is too small," the pronoun "it" refers to "X." The sentence is stating that "X" cannot fit inside "Y" due to its size being too small. Me: "This is incorrect, please give step by step logic on why you came to this conclusion" G4: Apologies for the confusion. I will provide a step-by-step logic for my interpretation: The sentence is: "The X doesn't fit in the Y because it is too small." We need to identify what "it" refers to in the sentence. There are two nouns in the sentence: X and Y. The sentence structure is such that the reason for X not fitting in Y is provided after the word "because." The sentence states that something is "too small," which is the reason for the lack of fit. Since the reason for the lack of fit is being given, we can infer that the pronoun "it" should be referring to one of the two nouns. However, upon closer examination, my previous interpretation was incorrect. Revisiting the analysis, the correct interpretation should be as follows: In the sentence "The X doesn't fit in the Y because it is too small," the pronoun "it" actually refers to "Y." The sentence is stating that "X" cannot fit inside "Y" because "Y" is too small to accommodate "X." My initial interpretation was wrong because I didn't consider that "it" could refer to "Y" and that the sentence is explaining why "X" cannot fit in "Y."