5 ms·

That looks cool, though I'd be very concerned about the possibility about ChatGPT "hallucinating" a `rm -rf /` or equivalent.

brokencode 4y ago
Just run all the git commands by another GPT bot that determines whether it would be harmful first. GPT bots all the way down.
- ipaddr 4y ago
  Can someone make a bot to verify the gpt bots are up and another to verify the first verifier is up
  - visarga 4y ago
    It's the same GPT but with a different prompt.
saghm 4y ago
Given that it is doing actual operations under the hood, it would be interesting to have it just output the "native" command for the operation rather than executing it. People could write wrappers around that output to execute it after prompting the user for confirmation or something like that so that users would have the choice to "opt into" the level of automatic execution they're comfortable with.
- danthelion 4y ago
  This way my approach when I created a similar tool, called git-genie[0]. It’s more of an educational tool first, explains the generared git command in detail. [0] https://github.com/danthelion/git-genie https://github.com/danthelion/git-genie
williamcotton 4y ago
[flagged]
- agolio 4y ago
  Perhaps explain to the person you are responding to why you disagree with them instead of making passive-aggressive jabs at their intellect
  - williamcotton 4y ago
    I don’t disagree with them. Reality disagrees with them. When LLMs “hallucinate” they don’t do so in random ways like somehow deciding to “rm -rf /“. They do so in predictable ways. What, am I supposed to handhold every person in these forums until they stop reading garbage on the internet and start either doing their own empirical research or at least read other empirical research? Edit: Apparently the answer is yes, I am supposed to handhold every person in these forums… and I will tirelessly do so, even in the face of provocatively incorrect yet populist statements like OPs… I still imagine I lose my patience every 100 or so instances, as seems about the average these days…
    - lolinder 4y ago
      The first two guidelines for this forum are these: > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. > Please don't fulminate. Please don't sneer, including at the rest of the community. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html If you get tired of respectfully explaining why people are wrong, then you can always stop commenting and go get some fresh air. What you should not do is lash out aggressively with zero explanation for why you feel so strongly about this.
      - williamcotton 4y ago
        Thanks, dad. Funny enough, I reviewed your comment history and you seem to be of the same opinion as the person I was responding to… …could that have perhaps motivated your patronizing tone?
williamcotton 4y ago
When LLMs “hallucinate” they don’t do so in random ways like somehow deciding to “rm -rf /“. They do so in predictable ways.
- bqmjjx0kac 4y ago
  Extraordinary claims require extraordinary evidence. Given that GPT3 has 175 billion nodes, how would you even begin to support the claim that it never (or sufficiently rarely) does things that are surprising to humans?
  - williamcotton 4y ago
    If you're looking for academic research, this is a great place to start: https://arxiv.org/abs/2202.03629 https://arxiv.org/abs/2202.03629 But you can get a general feel by using ChatGPT. Open up a new conversation and ask it something like, "What is the capital of France?". Note the response. Open up a new conversation and note the response. Soon enough you should be able to see that the responses are far from random. You can use the OpenAI APIs directly and have it run 10,000 or so iterations to see what kind of "hallucinations" it makes! They are not random!
    - Oxidation 4y ago
      Ask it details about a little-documented event and it'll happily tell you plausible, but utterly false, lies, however. Apparently the "early 2011 Bougainville earthquake" was magnitude 6.3, at a depth of 21.7km, on the 20th January and caused "widespread damage to buildings and infrastructure in the region, and triggered landslides that blocked roads and hampered rescue efforts". It was actually on the 7th Feb, a 6.4 and at a depth of 415km. There were "no immediate reports of damage or injuries". None of this is remotely surprising, considering it's a turbocharged statistical model and it probably ingested a few words about it at most, out of billions and billions, but somewhere along the line from "famous" to "footnote" subjects, it will segue into complete fiction.
      - 4y ago

brokencode 4y ago

Just run all the git commands by another GPT bot that determines whether it would be harmful first. GPT bots all the way down.

ipaddr 4y ago
Can someone make a bot to verify the gpt bots are up and another to verify the first verifier is up
- visarga 4y ago
  It's the same GPT but with a different prompt.

saghm 4y ago

Given that it is doing actual operations under the hood, it would be interesting to have it just output the "native" command for the operation rather than executing it. People could write wrappers around that output to execute it after prompting the user for confirmation or something like that so that users would have the choice to "opt into" the level of automatic execution they're comfortable with.

danthelion 4y ago
This way my approach when I created a similar tool, called git-genie[0]. It’s more of an educational tool first, explains the generared git command in detail. [0] https://github.com/danthelion/git-genie https://github.com/danthelion/git-genie

williamcotton 4y ago

[flagged]

agolio 4y ago
Perhaps explain to the person you are responding to why you disagree with them instead of making passive-aggressive jabs at their intellect
- williamcotton 4y ago
  I don’t disagree with them. Reality disagrees with them. When LLMs “hallucinate” they don’t do so in random ways like somehow deciding to “rm -rf /“. They do so in predictable ways. What, am I supposed to handhold every person in these forums until they stop reading garbage on the internet and start either doing their own empirical research or at least read other empirical research? Edit: Apparently the answer is yes, I am supposed to handhold every person in these forums… and I will tirelessly do so, even in the face of provocatively incorrect yet populist statements like OPs… I still imagine I lose my patience every 100 or so instances, as seems about the average these days…
  - lolinder 4y ago
    The first two guidelines for this forum are these: > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. > Please don't fulminate. Please don't sneer, including at the rest of the community. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html If you get tired of respectfully explaining why people are wrong, then you can always stop commenting and go get some fresh air. What you should not do is lash out aggressively with zero explanation for why you feel so strongly about this.
    - williamcotton 4y ago
      Thanks, dad. Funny enough, I reviewed your comment history and you seem to be of the same opinion as the person I was responding to… …could that have perhaps motivated your patronizing tone?

williamcotton 4y ago

When LLMs “hallucinate” they don’t do so in random ways like somehow deciding to “rm -rf /“. They do so in predictable ways.

bqmjjx0kac 4y ago
Extraordinary claims require extraordinary evidence. Given that GPT3 has 175 billion nodes, how would you even begin to support the claim that it never (or sufficiently rarely) does things that are surprising to humans?
- williamcotton 4y ago
  If you're looking for academic research, this is a great place to start: https://arxiv.org/abs/2202.03629 https://arxiv.org/abs/2202.03629 But you can get a general feel by using ChatGPT. Open up a new conversation and ask it something like, "What is the capital of France?". Note the response. Open up a new conversation and note the response. Soon enough you should be able to see that the responses are far from random. You can use the OpenAI APIs directly and have it run 10,000 or so iterations to see what kind of "hallucinations" it makes! They are not random!
  - Oxidation 4y ago
    Ask it details about a little-documented event and it'll happily tell you plausible, but utterly false, lies, however. Apparently the "early 2011 Bougainville earthquake" was magnitude 6.3, at a depth of 21.7km, on the 20th January and caused "widespread damage to buildings and infrastructure in the region, and triggered landslides that blocked roads and hampered rescue efforts". It was actually on the 7th Feb, a 6.4 and at a depth of 415km. There were "no immediate reports of damage or injuries". None of this is remotely surprising, considering it's a turbocharged statistical model and it probably ingested a few words about it at most, out of billions and billions, but somewhere along the line from "famous" to "footnote" subjects, it will segue into complete fiction.
    - 4y ago