4 ms·

This thread is already full of people throwing in their cheap takes by reading the headline or not even that. Listen to his answers. He's not making controvers

This thread is already full of people throwing in their cheap takes by reading the headline or not even that. Listen to his answers. He's not making controversial claims, excepting the one that LLMs teach us nothing about language. He says: - ChatGPT makes it easier to cheat. This is obviously true. - ChatGPT is a method of avoiding learning. In the context of the interview he's referring to its use not as an information retreival tool but as an actual essay writing tool. If we consider the latter, this is again obviously true. Not writing your essays teaches you nothing about essay writing, the topic at hand, and the process of thinking. - LLMs teach us basically nothing about language. Seems controversial, but I'm not an expert at all. - Education is in trouble because students are not engaged in education. We need education systems that are actually interesting and engaging. Again, this is obviously true. - It's not obvious what value ChatGPT has. Kind of controversial, but I think it's not actually obvious that ChatGPT is currently most used for funny Tweets, SEO optimization, and marketing tools. So I think he's not really saying much to get angry about here.

williamcotton 4y ago
That ChatGPT is good at truthfully answering one kind of prompt (analytic, where the facts are contained in the prompt, so the task is translation) and bad at truthfully answering another kind of prompt (synthetic, where the facts are not contained in the prompt, so the task is synthesis), tells me something about the nature of language. It turns out that setting up solutions to math problems is mainly a language translation task, followed by computation task as long as you have a way to eval() the resulting thunk: Do not perform calculations. Do not compute the answer. Use Javascript Math and Date to perform calculations. Wait for further questions. Show your work in the DESCRIPTION. The number of steps in the %%%THUNK%%% should match the number of steps in the %%%DESCRIPTION%%%. Always answer with Javascript compatible code in the %%%THUNK%%%, including numerical form without commas (eg, 238572348723). Always answer with this JSON compatible object form, eg, {"key":%%%VALUE%%%}: { "question": "4 days a week, Laura practices martial arts for 1.5 hours. Considering a week is 7 days, what is her average practice time per day each week?", "description":["Multiplying 1.5 and 4", "Dividing the step 1 answer by 7"], "thunk": "(function() { const step1 = 1.5 * 4; const answer = step1 / 7; return answer; })()" } Analytic augmentation is the procedure by which a question is augmented in a manner that promotes translation over synthesis. That this can be tested empirically is definitely something novel with regards to the philosophy of language!
- lamontcg 4y ago
  I've asked ChatGPT to take the derivative of an expression and the one it gave back to me was off by a minus sign. I tried to get it to calculate the expression for the height of a rocket flying vertically in constant gravity and the answer it gave me was just the rocket equation with h = instead of v = and I then got into a curious discussion with it about dimensional analysis where it just gaslit the crap out of me. If you think it is good at "analytic" tasks you haven't challenged it much.
  - williamcotton 4y ago
    And I’ve applied these principles of analytic augmentation to make the results of otherwise synthetic prompts more reliably truthful and in a testable and reproducible manner. You seem to be thrashing around wildly while using these tools and I can’t imagine why anyone would be interested in whatever conclusions result from such lack of methodology.
emodendroket 4y ago
Chomsky's universal grammar vs. the probabilistic model behind tools like ChatGPT or Google Translate is a pretty old debate.
- thundergolfer 4y ago
  Yes I know, that's why I said it seems controversial.
rvz 4y ago
> It's not obvious what value ChatGPT has. Kind of controversial, but I think it's not actually obvious that ChatGPT is currently most used for funny Tweets, SEO optimization, and marketing tools. So I think he's not really saying much to get angry about here. Or in other words, it is great at being a confident sophist and its use-case(s) are limited to almost nothing of serious use requiring explainability or trustworthiness; especially for legal, medical and financial applications or even search engines. If you cannot trust the output of an AI and it keeps hallucinating its answers, then what is the point? If OpenAI was being 'responsible' or following 'responsible AI' guidelines, maybe they should be writing the detectors first before releasing ChatGPT, GPT-3 into the wild? This AI hype cycle is no different to what happened with GPT-3 and it fizzled out just like how Clubhouse did. The only way is open source AI models.
ddren 4y ago
ChatGPT makes it easier to cheat, but cheating in that way was always an option, as long as you had enough money to pay someone else to do it for you. So in that sense, all that ChatGPT does is expose the flaws in the system by making this form of cheating available to everybody. I have to wonder why only now this is considered a serious problem that undermines education, but it was mostly ignored when the option was only available to the rich kids. I think this can only solved by having a clear separation between learning and obtaining the credentials that certify that you possess the knowledge that it is expected of you to do some job. Universities should be centers of learning and students should be writing their essays because they recognize the value that doing so provides to them, not just because they need a certificate issued by the university to apply to the jobs that they desire. The certificate part should come from elsewhere, probably through means that make cheating much harder, such as exams and individual interviews. This would also open the door to other avenues of learning that may be more suited for some people, ie. self-taught people may have more success learning from a book or an online resource than from sitting through several hours of lectures every day.
- thundergolfer 4y ago
  Yes, there were always rich kids that were able to cheat, using college just for the status and credentialism, and there still are. What Chomsky is interested in, and what we should be interested in, is how we get rich kids like J. Robert Oppenheimer. Oppenheimer didn't need to become educated in and world class at physics, be he did it anyway, and it's probably the lack of need which increased his interest and ability. I think your separation idea is interesting. At first blush, it seems a good move to keep examination as far away from education as possible, as they're very different things.
- bumby 4y ago
  >I have to wonder why only now this is considered a serious problem that undermines education, but it was mostly ignored when the option was only available to the rich kids. I think you've answered your own question: it's a matter of scale. Systems can tolerate a certain proportion of bad actors but there is an inevitable tipping point where the number of bad actors grows to a point where the system becomes unstable. But I 100% agree that university credentials are only a very rough proxy for competence. I also agree that internal motivation is better than external motivation. Your suggestion is seen in other professions like medicine or capital "E" Engineering, where there are boards/exams and continuing education to certify competence in the form of licensure. This comes with its own problems, though, like how the licensing boards can become a cabal to protect their own self interests (see the complaints about the AMA limiting doctor licensure in an effort to maintain higher pay for physicians). It also gives more leverage to the individual, which many industry lobbyists would probably be against.
dwaltrip 4y ago
> It's not obvious what value ChatGPT has. This is a blatantly incorrect take. Anyone who can’t see that ChatGPT has real value has blinders on. It has severe limitations and many quirks, but millions of people are getting value from it. I use it regularly while developing software. If you learn how to leverage its strengths and are aware of the pitfalls, it isn’t too hard to get value out of. It is also a good investment to learn how to use these tools, as they will become much more capable and useful over time.
kledru 4y ago
This seems mostly right. But ChatGPT also can make it a lot easier to learn if one wants to learn. But I think that you need to be able to critically evaluate it's output for that. And to formulate your ideas on some level. If one never obtains those skills, perhaps learning with chatGPT won't be of much use.
- elforce002 4y ago
  I beg to differ. You can try to learn, sure, but it's tricky, to say the least. You learn from people with years of XP in the subject, many references, among other things. The knowledge you gain from them is vetted by pretty much the entire scientific community. If you're not an expert in any field, chatGPT can give you any answer and you'll take it as good and valid. That's a dangerous precedent there. And just in case you said: "you can double check what chatGPT is throwing at me", well, that's wishful thinking. People hardly go the extra mile, less of all, when studying is involved. It's difficult to just concentrate in one subject, let alone the whole curriculum, and that's with guidance. What type of guidance does chatGPT offer? How are you going to measure the student? If he/she fails, what are the options? There are lots of questions to be answered before we incorporate chatGPT the the current education system. I'm not saying we couldn't do it, I'm just saying that while tech is there to help us, this time around is dangerous because it can do more harm than good, and could, in fact, set humanity back if mishandled.
  - ausbah 4y ago
    I think the value that chatgpt could (emphasis on could) provide as a learning tool is the same value that a repl provides to coding, instant feedback in a playground environment. I think it would be immensely useful or anyone learning anything to be able to say "I don't stand specific thing based on prior conversation please elaborate", or "give me a new example of specific thing" or "explain my why my thought process about X thing is wrong". would really boost engagement with learning a think by lowering the barrier to getting stuck on something, if all the BSing and incorrect info it has could be worked out

williamcotton 4y ago

That ChatGPT is good at truthfully answering one kind of prompt (analytic, where the facts are contained in the prompt, so the task is translation) and bad at truthfully answering another kind of prompt (synthetic, where the facts are not contained in the prompt, so the task is synthesis), tells me something about the nature of language. It turns out that setting up solutions to math problems is mainly a language translation task, followed by computation task as long as you have a way to eval() the resulting thunk: Do not perform calculations. Do not compute the answer. Use Javascript Math and Date to perform calculations. Wait for further questions. Show your work in the DESCRIPTION. The number of steps in the %%%THUNK%%% should match the number of steps in the %%%DESCRIPTION%%%. Always answer with Javascript compatible code in the %%%THUNK%%%, including numerical form without commas (eg, 238572348723). Always answer with this JSON compatible object form, eg, {"key":%%%VALUE%%%}: { "question": "4 days a week, Laura practices martial arts for 1.5 hours. Considering a week is 7 days, what is her average practice time per day each week?", "description":["Multiplying 1.5 and 4", "Dividing the step 1 answer by 7"], "thunk": "(function() { const step1 = 1.5 * 4; const answer = step1 / 7; return answer; })()" } Analytic augmentation is the procedure by which a question is augmented in a manner that promotes translation over synthesis. That this can be tested empirically is definitely something novel with regards to the philosophy of language!

lamontcg 4y ago
I've asked ChatGPT to take the derivative of an expression and the one it gave back to me was off by a minus sign. I tried to get it to calculate the expression for the height of a rocket flying vertically in constant gravity and the answer it gave me was just the rocket equation with h = instead of v = and I then got into a curious discussion with it about dimensional analysis where it just gaslit the crap out of me. If you think it is good at "analytic" tasks you haven't challenged it much.
- williamcotton 4y ago
  And I’ve applied these principles of analytic augmentation to make the results of otherwise synthetic prompts more reliably truthful and in a testable and reproducible manner. You seem to be thrashing around wildly while using these tools and I can’t imagine why anyone would be interested in whatever conclusions result from such lack of methodology.

emodendroket 4y ago

Chomsky's universal grammar vs. the probabilistic model behind tools like ChatGPT or Google Translate is a pretty old debate.

thundergolfer 4y ago
Yes I know, that's why I said it seems controversial.

rvz 4y ago

> It's not obvious what value ChatGPT has. Kind of controversial, but I think it's not actually obvious that ChatGPT is currently most used for funny Tweets, SEO optimization, and marketing tools. So I think he's not really saying much to get angry about here. Or in other words, it is great at being a confident sophist and its use-case(s) are limited to almost nothing of serious use requiring explainability or trustworthiness; especially for legal, medical and financial applications or even search engines. If you cannot trust the output of an AI and it keeps hallucinating its answers, then what is the point? If OpenAI was being 'responsible' or following 'responsible AI' guidelines, maybe they should be writing the detectors first before releasing ChatGPT, GPT-3 into the wild? This AI hype cycle is no different to what happened with GPT-3 and it fizzled out just like how Clubhouse did. The only way is open source AI models.

ddren 4y ago

ChatGPT makes it easier to cheat, but cheating in that way was always an option, as long as you had enough money to pay someone else to do it for you. So in that sense, all that ChatGPT does is expose the flaws in the system by making this form of cheating available to everybody. I have to wonder why only now this is considered a serious problem that undermines education, but it was mostly ignored when the option was only available to the rich kids. I think this can only solved by having a clear separation between learning and obtaining the credentials that certify that you possess the knowledge that it is expected of you to do some job. Universities should be centers of learning and students should be writing their essays because they recognize the value that doing so provides to them, not just because they need a certificate issued by the university to apply to the jobs that they desire. The certificate part should come from elsewhere, probably through means that make cheating much harder, such as exams and individual interviews. This would also open the door to other avenues of learning that may be more suited for some people, ie. self-taught people may have more success learning from a book or an online resource than from sitting through several hours of lectures every day.

thundergolfer 4y ago
Yes, there were always rich kids that were able to cheat, using college just for the status and credentialism, and there still are. What Chomsky is interested in, and what we should be interested in, is how we get rich kids like J. Robert Oppenheimer. Oppenheimer didn't need to become educated in and world class at physics, be he did it anyway, and it's probably the lack of need which increased his interest and ability. I think your separation idea is interesting. At first blush, it seems a good move to keep examination as far away from education as possible, as they're very different things.
bumby 4y ago
>I have to wonder why only now this is considered a serious problem that undermines education, but it was mostly ignored when the option was only available to the rich kids. I think you've answered your own question: it's a matter of scale. Systems can tolerate a certain proportion of bad actors but there is an inevitable tipping point where the number of bad actors grows to a point where the system becomes unstable. But I 100% agree that university credentials are only a very rough proxy for competence. I also agree that internal motivation is better than external motivation. Your suggestion is seen in other professions like medicine or capital "E" Engineering, where there are boards/exams and continuing education to certify competence in the form of licensure. This comes with its own problems, though, like how the licensing boards can become a cabal to protect their own self interests (see the complaints about the AMA limiting doctor licensure in an effort to maintain higher pay for physicians). It also gives more leverage to the individual, which many industry lobbyists would probably be against.

dwaltrip 4y ago

> It's not obvious what value ChatGPT has. This is a blatantly incorrect take. Anyone who can’t see that ChatGPT has real value has blinders on. It has severe limitations and many quirks, but millions of people are getting value from it. I use it regularly while developing software. If you learn how to leverage its strengths and are aware of the pitfalls, it isn’t too hard to get value out of. It is also a good investment to learn how to use these tools, as they will become much more capable and useful over time.

kledru 4y ago

This seems mostly right. But ChatGPT also can make it a lot easier to learn if one wants to learn. But I think that you need to be able to critically evaluate it's output for that. And to formulate your ideas on some level. If one never obtains those skills, perhaps learning with chatGPT won't be of much use.

elforce002 4y ago
I beg to differ. You can try to learn, sure, but it's tricky, to say the least. You learn from people with years of XP in the subject, many references, among other things. The knowledge you gain from them is vetted by pretty much the entire scientific community. If you're not an expert in any field, chatGPT can give you any answer and you'll take it as good and valid. That's a dangerous precedent there. And just in case you said: "you can double check what chatGPT is throwing at me", well, that's wishful thinking. People hardly go the extra mile, less of all, when studying is involved. It's difficult to just concentrate in one subject, let alone the whole curriculum, and that's with guidance. What type of guidance does chatGPT offer? How are you going to measure the student? If he/she fails, what are the options? There are lots of questions to be answered before we incorporate chatGPT the the current education system. I'm not saying we couldn't do it, I'm just saying that while tech is there to help us, this time around is dangerous because it can do more harm than good, and could, in fact, set humanity back if mishandled.
- ausbah 4y ago
  I think the value that chatgpt could (emphasis on could) provide as a learning tool is the same value that a repl provides to coding, instant feedback in a playground environment. I think it would be immensely useful or anyone learning anything to be able to say "I don't stand specific thing based on prior conversation please elaborate", or "give me a new example of specific thing" or "explain my why my thought process about X thing is wrong". would really boost engagement with learning a think by lowering the barrier to getting stuck on something, if all the BSing and incorrect info it has could be worked out