4 ms·
There seems to be a profound lack of scientific rigour and methodology amongst the engineers who build this stuff. The engineer's reply is always, "but the outp
by mjburgess 3y ago
There seems to be a profound lack of scientific rigour and methodology amongst the engineers who build this stuff. The engineer's reply is always, "but the output works".
But that's not what engineers always say, esp. when they're on company boards. They often make extraordinary claims about how it works -- and then get annoyed when people do actual experiments.
This is a paper about what the properties of the system are, not the properties of its output. To determine these properties you need a ruthless, scientific focus, on the null (/failure) cases.
Many of the hypotheses given by hyping engineers are just that, and often trivial to disprove with papers such as this.
- YetAnotherNick 3y ago> They often make extraordinary claims about how it works I often hear X making wrong claims, but never seen anyone making it. Could you give any actual examples of someone making extraordinary claims? OpenAI is very clear that chatGPT could produce incorrect output. I heavily utilize GPT 4 and copilot and while it helps me a lot, I am actually very aware its output needs to be verified. Same with Stackoverflow, where a lot of the answers are just plain wrong and it needs to be verified. Part of what I am beginning to understand the scenarios in which output could be trusted.
- infinghxsg 3y ago[dead]
- xcv123 3y agoBut the paper is bullshit. Their task is easily solved by GPT-4. https://chat.openai.com/share/a28deca2-b989-4029-b042-b8434b80aaf6 https://chat.openai.com/share/a28deca2-b989-4029-b042-b8434b...
- suby 3y agoInteresting. You said Gpt4, but that link shows ChatGpt's avatar as 3.5's. And I'm convinced you were using 4 as I just tried replicating this; with 3.5 it fails and with 4 it succeeds. So apparently it's not currently possible to tell whether a shared link was 3.5 or 4? Unfortunate if so.
- xcv123 3y agoThe shared link was from my own GPT-4 session. It shows "Model: GPT-4" in the header of my chat from which I shared the link.
- elcomet 3y agoNo it shows Model: Default
- xcv123 3y agoYes but it's from GPT-4. "Model: Default" is a defect in their link sharing. In my private session it says "Model: GPT-4"
- lordnacho 3y agoHow do you set it to 4? I can only find 3.5 somehow.
- xcv123 3y agoYou must pay for it. ChatGPT Plus. $20 per month.
- twright0 3y agoSometimes! My attempt with GPT-4 yields a response where it acknowledges the print/len swap, but does not produce correct code in the end - it sort of loses track of what the original goal was. https://chat.openai.com/share/300382cb-ac72-4a75-847c-ecbf5ab83720 https://chat.openai.com/share/300382cb-ac72-4a75-847c-ecbf5a...
- mjburgess 3y agoAnd if we're doing Science, ie., trying to explain how ChatGPT works and what it's intrinsic properties are --- this case is far more significant than the other. Inasmuch as the hypothesis that ChatGPT works "so as to be actually sensitive to the meaning of the code" is here falsified -- by a single case. An infinite number of apparent confirmations of this hypothesis are now Invalid!
- fiso64 3y agoHumans make foolish, nonsensical mistakes too. Does that mean that they're not intelligent and not sensitive to the meaning of the code?
- YeGoblynQueenne 3y agoWe already know that humans are sensitive to meaning. That's not a hypothesis that needs testing.
- TeMPOraL 3y agoThis experiment only proves LLMs are sensitive to meaning too. It was also already known.
- YeGoblynQueenne 3y agoAll I can see with any clarity is that LLMs are sensitive to the user's prompt.
- 3y ago
- borissk 3y ago"ruthless, scientific focus, on the null (/failure) cases" means you're not going to build anything of value ever. There is risk in every startup, innovation, new technology, etc and a big part of making those work is laser focusing on success and leaving the failure cases to someone else (once you have a successful product and/or business).
- YeGoblynQueenne 3y agoThat's why a) nothing ever works and b) business is not science.