4 ms·

Code Interpreter will be available to all ChatGPT Plus users over the next week

54 points by maxbaines 3y ago

skilled 3y ago
People who have had access to this already, can you share some interesting use cases?
- jpdus 3y ago
  A very good article on the (early) capabilities by Ethan Mollick: https://www.oneusefulthing.org/p/it-is-starting-to-get-strange https://www.oneusefulthing.org/p/it-is-starting-to-get-stran...
  - eterps 3y ago
    Thanks, that's really helpful
  - DonaldPShimoda 3y ago
    From your linked article: > So the AI shows genuine creativity in problem solving. This is garbage. The AI does not demonstrate "genuine creativity". LLMs generate statistically likely sequences of tokens based on their training data, which in this case is more or less the entire internet. The author asserts with absolutely no citation: > I figured no one had ever made a request like that before. What an idiotic claim. You're talking about the internet, for crying out loud. It's almost guaranteed somebody has written something like what you're writing before when it comes to requests of this nature. Is the technology cool? Yes. Is it cooler than similar technologies from a year ago, or even a few months ago? Sure. But it's not intelligent. It doesn't think; it doesn't know. It generates statistically likely sequences of tokens for any input, and nothing more. People need to stop pretending otherwise.
    - deleted 3y ago
      [deleted]
    - Baeocystin 3y ago
      >What an idiotic claim. You're talking about the internet, for crying out loud. It's almost guaranteed somebody has written something like what you're writing before when it comes to requests of this nature. I wonder if this is as true as we collectively think? ~15% of Google searches are unique†. Informally, folks I know that work on other search engines see even higher percentages, even after decades+ of history. † https://www.seroundtable.com/15-percent-search-queries-google-new-32936.html https://www.seroundtable.com/15-percent-search-queries-googl...
    - climatologist 3y ago
      I get what you're saying but people don't really care. The typical/average person does not know anything about derivatives, backpropagation, or probabilities so to them it all seems like magic and they anthropomorphize what they're seeing as something intelligent. Some folks that know how this stuff works and do a good job of explaining the limitations are Melanie Mitchel and Francois Chollet. Both have extensive experience in the field and have also written books on AI. You can spend your time trying to explain to every random person that computers can't think but they're not gonna understand what you're saying because to them it seems like a large enough Markov chain is actually thinking.
      - selalipop 3y ago
        What about those of us who do understand them and just don't agree? After all you could simplify it to a layperson as: 'the LLM is just doing fancy autocomplete based on how stuff appeared in the training data so that means they're not creative' The first part is not really up for debate, but it's the second part is where some of us disagree. Creativity doesn't mean novel in existence, it means novel within some context: https://www.researchgate.net/publication/254301596_The_Standard_Definition_of_Creativity https://www.researchgate.net/publication/254301596_The_Stand... — At some point, the push back against these models being creative starts to feel like it's just as emotion driven as the people who are over-anthropomorphizing the models: "If I accept something I know is just a ball of linear algebra is creative, then it's cheapening the definition of creativity." People bring up the stochastic parrot argument forgetting that the original paper was predicated on the dangers of not considering the power that lies in something that's "just" a stochastic parrot.
        climatologist 3y ago
        Ask it to solve sudoku and report back. Not generate code but actually solve a puzzle in the prompt.
        selalipop 3y ago
        This is what I mean when I say "inverse-anthropomorphization" crowd is increasingly emotion over facts. My reply to you was predicated on a compilation of centuries of scientific study on the subject of creativity. Your knee-jerk reply is to proclaim it's bad at sudoku while going out of your way to place artificial constraints on it. Touting its inability to solve sudoku in-context feels like a slightly hamfisted way of saying it's a probability based model operating on tokens but like I said before, there are plenty of us who already understand that. We also realize that you can find arbitrary gaps in any sufficiently complex system. You didn't even need to rely on such a specific example, you could have touted any number of variations on common logic puzzles that they just fall completely on their faces for. Gaps aren't damning until you tie them to what you want out of the system. The LLM can be bad at Sudoku and capable of creativity in some domain. It's more useful to explore unexpected properties of a complex system than it is to parade things that the system is already expected to be bad at.
    - ghotli 3y ago
      I ultimately don't really disagree with you but your vitriol really undermines your point. Figured you should be aware of how you come across, at least from my perspective. Plus there's an argument to be made that you and I are also generating likely sequences based on the sum of our experiences just in a more sophisticated (and evolutionarily refined) manner. There's at least no hard evidence to the contrary that I'm aware of. Fake edit: I'm aware that my final statement is somewhat bait for the neuroscientists in the crowd. In this context I'm of the opinion that defining if agi 'thinks' requires a far more rigorous definition of what it is that we (and other problem solving / pass-the-mirror-test animals) are doing. For example, "It's the soul and GPT-4 doesn't have one" doesn't even come close to being rigorous enough, imo.
- veb 3y ago
  Here's one involving Diablo 4 - https://www.youtube.com/watch?v=1IxuAyYocoQ https://www.youtube.com/watch?v=1IxuAyYocoQ
- gamegoblin 3y ago
  It's about as useful as having an obedient 2nd year CS student with access to a juypter notebook. Which is to say, pretty useful for certain stuff! You can do flows like: - Upload CSV of stock data - "Graph the closing price each day" - "On the other axis, add a line showing volatility" - "Filter out to just days with above-average volatility" etc Pretty useful for analyzing raw data.
- mitchellpkt 3y ago
  For me, the biggest productivity boost is from work that is (1) boring or time consuming to do, but (2) easy to verify / QC. (3) Extra useful if it is something that can the code interpreter can unit test in place, and iterate the code until the test passes. For example, I’ll show it some big nested json response, then ask: Please write a python helper function get_foo_and_bar(endpoint: str = “whatever”)->Dict[str, Any] that hits the endpoint with <parameters>, then pulls out the foo and bar, and converts bar to List[int]. Please write and run a test that verifies the retrieved values from the above response are foo=“baz” and bar=[6,9,2]. It might take it 2 or 3 tries to get right, but it will iterate on its own until it works. Same process as if I were writing but by hand, but the 2-3 iterations might take me a few minutes and only takes the model a few seconds.
- synaesthesisx 3y ago
  Code Interpreter is great for "quick and dirty" data analysis & visualizations. I've seen it struggle with larger datasets (not sure what the current limit is) but I can certainly see this sort of stuff replacing basic BI at organizations. Microsoft is working on embedding this sort of stuff into what they call Fabric - which is essentially meant to be PowerBI on steroids. I know in the current preview, you can interact with data using a chat-like interface ("Show me our revenue for the last 12 months sorted by customer") .
  - zurfer 3y ago
    One large use case is certainly data analysis. But one big challenge with analytics is: you always get a result. Wrong filter, wrong aggregation function, wrong table or wrong join can all still result in plausible but wrong results. Hallucinations take the form of arbitrary logic. Disclaimer: I'm a cofounder at getdot.ai. We are starting to solve that with a semantic layer, but for now it is limited to business reporting (sales per territory, etc.). PowerBI will imo make it even easier for people to create visuals (NL vs drag n drop), but right now fails at the consistency part.

54 points by maxbaines 3y ago

skilled 3y ago

People who have had access to this already, can you share some interesting use cases?

jpdus 3y ago
A very good article on the (early) capabilities by Ethan Mollick: https://www.oneusefulthing.org/p/it-is-starting-to-get-strange https://www.oneusefulthing.org/p/it-is-starting-to-get-stran...
- eterps 3y ago
  Thanks, that's really helpful
- DonaldPShimoda 3y ago
  From your linked article: > So the AI shows genuine creativity in problem solving. This is garbage. The AI does not demonstrate "genuine creativity". LLMs generate statistically likely sequences of tokens based on their training data, which in this case is more or less the entire internet. The author asserts with absolutely no citation: > I figured no one had ever made a request like that before. What an idiotic claim. You're talking about the internet, for crying out loud. It's almost guaranteed somebody has written something like what you're writing before when it comes to requests of this nature. Is the technology cool? Yes. Is it cooler than similar technologies from a year ago, or even a few months ago? Sure. But it's not intelligent. It doesn't think; it doesn't know. It generates statistically likely sequences of tokens for any input, and nothing more. People need to stop pretending otherwise.
  - deleted 3y ago
    [deleted]
  - Baeocystin 3y ago
    >What an idiotic claim. You're talking about the internet, for crying out loud. It's almost guaranteed somebody has written something like what you're writing before when it comes to requests of this nature. I wonder if this is as true as we collectively think? ~15% of Google searches are unique†. Informally, folks I know that work on other search engines see even higher percentages, even after decades+ of history. † https://www.seroundtable.com/15-percent-search-queries-google-new-32936.html https://www.seroundtable.com/15-percent-search-queries-googl...
  - climatologist 3y ago
    I get what you're saying but people don't really care. The typical/average person does not know anything about derivatives, backpropagation, or probabilities so to them it all seems like magic and they anthropomorphize what they're seeing as something intelligent. Some folks that know how this stuff works and do a good job of explaining the limitations are Melanie Mitchel and Francois Chollet. Both have extensive experience in the field and have also written books on AI. You can spend your time trying to explain to every random person that computers can't think but they're not gonna understand what you're saying because to them it seems like a large enough Markov chain is actually thinking.
    - selalipop 3y ago
      What about those of us who do understand them and just don't agree? After all you could simplify it to a layperson as: 'the LLM is just doing fancy autocomplete based on how stuff appeared in the training data so that means they're not creative' The first part is not really up for debate, but it's the second part is where some of us disagree. Creativity doesn't mean novel in existence, it means novel within some context: https://www.researchgate.net/publication/254301596_The_Standard_Definition_of_Creativity https://www.researchgate.net/publication/254301596_The_Stand... — At some point, the push back against these models being creative starts to feel like it's just as emotion driven as the people who are over-anthropomorphizing the models: "If I accept something I know is just a ball of linear algebra is creative, then it's cheapening the definition of creativity." People bring up the stochastic parrot argument forgetting that the original paper was predicated on the dangers of not considering the power that lies in something that's "just" a stochastic parrot.
      - climatologist 3y ago
        Ask it to solve sudoku and report back. Not generate code but actually solve a puzzle in the prompt.
        selalipop 3y ago
        This is what I mean when I say "inverse-anthropomorphization" crowd is increasingly emotion over facts. My reply to you was predicated on a compilation of centuries of scientific study on the subject of creativity. Your knee-jerk reply is to proclaim it's bad at sudoku while going out of your way to place artificial constraints on it. Touting its inability to solve sudoku in-context feels like a slightly hamfisted way of saying it's a probability based model operating on tokens but like I said before, there are plenty of us who already understand that. We also realize that you can find arbitrary gaps in any sufficiently complex system. You didn't even need to rely on such a specific example, you could have touted any number of variations on common logic puzzles that they just fall completely on their faces for. Gaps aren't damning until you tie them to what you want out of the system. The LLM can be bad at Sudoku and capable of creativity in some domain. It's more useful to explore unexpected properties of a complex system than it is to parade things that the system is already expected to be bad at.
  - ghotli 3y ago
    I ultimately don't really disagree with you but your vitriol really undermines your point. Figured you should be aware of how you come across, at least from my perspective. Plus there's an argument to be made that you and I are also generating likely sequences based on the sum of our experiences just in a more sophisticated (and evolutionarily refined) manner. There's at least no hard evidence to the contrary that I'm aware of. Fake edit: I'm aware that my final statement is somewhat bait for the neuroscientists in the crowd. In this context I'm of the opinion that defining if agi 'thinks' requires a far more rigorous definition of what it is that we (and other problem solving / pass-the-mirror-test animals) are doing. For example, "It's the soul and GPT-4 doesn't have one" doesn't even come close to being rigorous enough, imo.
veb 3y ago
Here's one involving Diablo 4 - https://www.youtube.com/watch?v=1IxuAyYocoQ https://www.youtube.com/watch?v=1IxuAyYocoQ
gamegoblin 3y ago
It's about as useful as having an obedient 2nd year CS student with access to a juypter notebook. Which is to say, pretty useful for certain stuff! You can do flows like: - Upload CSV of stock data - "Graph the closing price each day" - "On the other axis, add a line showing volatility" - "Filter out to just days with above-average volatility" etc Pretty useful for analyzing raw data.
mitchellpkt 3y ago
For me, the biggest productivity boost is from work that is (1) boring or time consuming to do, but (2) easy to verify / QC. (3) Extra useful if it is something that can the code interpreter can unit test in place, and iterate the code until the test passes. For example, I’ll show it some big nested json response, then ask: Please write a python helper function get_foo_and_bar(endpoint: str = “whatever”)->Dict[str, Any] that hits the endpoint with <parameters>, then pulls out the foo and bar, and converts bar to List[int]. Please write and run a test that verifies the retrieved values from the above response are foo=“baz” and bar=[6,9,2]. It might take it 2 or 3 tries to get right, but it will iterate on its own until it works. Same process as if I were writing but by hand, but the 2-3 iterations might take me a few minutes and only takes the model a few seconds.
synaesthesisx 3y ago
Code Interpreter is great for "quick and dirty" data analysis & visualizations. I've seen it struggle with larger datasets (not sure what the current limit is) but I can certainly see this sort of stuff replacing basic BI at organizations. Microsoft is working on embedding this sort of stuff into what they call Fabric - which is essentially meant to be PowerBI on steroids. I know in the current preview, you can interact with data using a chat-like interface ("Show me our revenue for the last 12 months sorted by customer") .
- zurfer 3y ago
  One large use case is certainly data analysis. But one big challenge with analytics is: you always get a result. Wrong filter, wrong aggregation function, wrong table or wrong join can all still result in plausible but wrong results. Hallucinations take the form of arbitrary logic. Disclaimer: I'm a cofounder at getdot.ai. We are starting to solve that with a semantic layer, but for now it is limited to business reporting (sales per territory, etc.). PowerBI will imo make it even easier for people to create visuals (NL vs drag n drop), but right now fails at the consistency part.