5 ms·

Aside from the social risks and centralization of control, I think the rush to convert everything to LLMs puts quite a bit of our progress in computer science a

by lsy 3y ago

Aside from the social risks and centralization of control, I think the rush to convert everything to LLMs puts quite a bit of our progress in computer science at risk should it prove to be successful. Over the last several decades we've built up more complex and precise notation and systems to allow for repeatability, reliability, composability, etc. And large companies are currently attempting to trash all of the field's progress on notation in favor of talking to the computer in English (a rich but highly error-prone language) and then having the computer do impenetrable transformations guided only by pretty-please prompts with the hope that some high percentage of the time the result will be usable.

kjkjadksj 3y ago
I don’t think any company is seriously considering that, having played with chatgpt for a bit. Getting it to generate functional code requires so much guardrailing and validating the output that you basically need the domain knowledge anyhow as well as the ability to code to even use ai well as it is. I don’t believe it will ever get to a state where it can just be trusted to write good code for a complicated purpose without a skilled human shepherding it all along the way. The problems we solve are too bespoke a lot of the times. Its just a problem of how these models are fundamentally constructed: train on data and project new data onto it. Problems arise when you understand training sets are finite and contain things that already exist.
- js8 3y ago
  No, it will be more subtle. The features you will get in software products, as a user, will get AI-ified. Mostly for cost reasons. What it means, instead of building a proper feature that works correctly and handles all the edge cases, AI will be invoked to resolve the discrepancies and "make it work". It will mostly work, but will be less reliable and understandable. The end result will be more shifting of burden of errors from the software companies to users.
  - RecycledEle 3y ago
    That is just dystopian enough to be believable.
  - xulres 3y ago
    If I see the crunch at my own corp and of friends. We are at the point we're we just make it work.
ad404b8a372f2b9 3y ago
Reminds me of this episode of star trek: https://en.wikipedia.org/wiki/When_the_Bough_Breaks_(Star_Trek:_The_Next_Generation) https://en.wikipedia.org/wiki/When_the_Bough_Breaks_(Star_Tr... It's about an ancient civilization so advanced they have tools and computers that can create art just from their intent. But they have grown so distant from the inner workings of these tools that they are incapable of modifying or repairing them and their survival is put into jeopardy because of it.
- ravenstine 3y ago
  For as substandard as most of the season 1 TNG episodes were, there was a lot of interesting ideas sprinkled here and there like the one you mentioned. It's too bad that poor acting and the ridiculously small scale of the conflict in the episode distracted from things like that. They probably should have abandoned the whole "we can't reproduce so let's kidnap children" thing for contrasting that civilization's relationship to technology with that of Earth's. IMO, there's already a moral dilemma in whether a failing technological civilization should be taught how to reproduce by outsiders whose relationship with technology is successful.
  - kristofferg 3y ago
    Sub what “standard”? I, for one, find the “mixed bag of soapy philosophy” a timeless treat. Not including the later Enterprise and the JJA movies.
    - dekhn 3y ago
      I think the standard was set in the last few seasons of TNG and the entirety of DS9. The crew gelled better and the topics were more weighty, yet human. I think probably it was around the time that Picard became Locutus and then the cardassian war
- aaroninsf 3y ago
  This is us now if you squint. Survival risk, maybe not, but we are certain to see massive die-off and a decades-long cold start given a perturbation maybe an order larger than the COVID pandemic, when the system is already undergoing unprecedented stress from climate change, as appears likely in the coming century...
cptcobalt 3y ago
I think you need both! Our underlying data should still be extremely principled and structured. But I think LLMs enable significantly better human computer interaction, where humans aren't interacting directly with the data in your preferred manner. For example, I've been parsing sentiment across the internet from the app I run, and using it to crunch it down into a very structured internal tracking database I have. I want to canvass as many places as possible. Using an LLM for that transformation has been stellar, and I'd casually say I'm at around 90% accuracy, which is more than sufficient for my planning & justification needs. In that sense, it should just be an additional tool in a toolbox, not a platform to bet the house on.
- JTyQZSnP3cQGa8B 3y ago
  > LLMs enable significantly better human computer interaction, where humans aren't interacting directly with the data in your preferred manner. It doesn’t explain why a randomized black box is needed. It only shows that we need better tools (i.e. not randomized and open).
deleted 3y ago
[deleted]
lukev 3y ago
I agree completely. Having LLMs use programming languages/tools/libraries built for humans is the wrong path to fully leverage their potential. They are capable of so much, but trying to use them as a drop-in replacement for humans isn't the best use for them. Frankly, it's impressive it works at all, but I think we will find hard limits on the actual complexity and reliability of the systems they can build in this manner. We still need intelligent, rigorous system design. LLMs have the strong potential to let system builders operate at a higher level of abstraction, but in my opinion having LLMs write Python or SQL is an evolutionary dead end. My own work is directly focused on this and what IS the proper place of a LLM in a data processing or knowledge system.

by lsy 3y ago

kjkjadksj 3y ago

I don’t think any company is seriously considering that, having played with chatgpt for a bit. Getting it to generate functional code requires so much guardrailing and validating the output that you basically need the domain knowledge anyhow as well as the ability to code to even use ai well as it is. I don’t believe it will ever get to a state where it can just be trusted to write good code for a complicated purpose without a skilled human shepherding it all along the way. The problems we solve are too bespoke a lot of the times. Its just a problem of how these models are fundamentally constructed: train on data and project new data onto it. Problems arise when you understand training sets are finite and contain things that already exist.

js8 3y ago
No, it will be more subtle. The features you will get in software products, as a user, will get AI-ified. Mostly for cost reasons. What it means, instead of building a proper feature that works correctly and handles all the edge cases, AI will be invoked to resolve the discrepancies and "make it work". It will mostly work, but will be less reliable and understandable. The end result will be more shifting of burden of errors from the software companies to users.
- RecycledEle 3y ago
  That is just dystopian enough to be believable.
- xulres 3y ago
  If I see the crunch at my own corp and of friends. We are at the point we're we just make it work.

ad404b8a372f2b9 3y ago

Reminds me of this episode of star trek: https://en.wikipedia.org/wiki/When_the_Bough_Breaks_(Star_Trek:_The_Next_Generation) https://en.wikipedia.org/wiki/When_the_Bough_Breaks_(Star_Tr... It's about an ancient civilization so advanced they have tools and computers that can create art just from their intent. But they have grown so distant from the inner workings of these tools that they are incapable of modifying or repairing them and their survival is put into jeopardy because of it.

ravenstine 3y ago
For as substandard as most of the season 1 TNG episodes were, there was a lot of interesting ideas sprinkled here and there like the one you mentioned. It's too bad that poor acting and the ridiculously small scale of the conflict in the episode distracted from things like that. They probably should have abandoned the whole "we can't reproduce so let's kidnap children" thing for contrasting that civilization's relationship to technology with that of Earth's. IMO, there's already a moral dilemma in whether a failing technological civilization should be taught how to reproduce by outsiders whose relationship with technology is successful.
- kristofferg 3y ago
  Sub what “standard”? I, for one, find the “mixed bag of soapy philosophy” a timeless treat. Not including the later Enterprise and the JJA movies.
  - dekhn 3y ago
    I think the standard was set in the last few seasons of TNG and the entirety of DS9. The crew gelled better and the topics were more weighty, yet human. I think probably it was around the time that Picard became Locutus and then the cardassian war
aaroninsf 3y ago
This is us now if you squint. Survival risk, maybe not, but we are certain to see massive die-off and a decades-long cold start given a perturbation maybe an order larger than the COVID pandemic, when the system is already undergoing unprecedented stress from climate change, as appears likely in the coming century...

cptcobalt 3y ago

I think you need both! Our underlying data should still be extremely principled and structured. But I think LLMs enable significantly better human computer interaction, where humans aren't interacting directly with the data in your preferred manner. For example, I've been parsing sentiment across the internet from the app I run, and using it to crunch it down into a very structured internal tracking database I have. I want to canvass as many places as possible. Using an LLM for that transformation has been stellar, and I'd casually say I'm at around 90% accuracy, which is more than sufficient for my planning & justification needs. In that sense, it should just be an additional tool in a toolbox, not a platform to bet the house on.

JTyQZSnP3cQGa8B 3y ago
> LLMs enable significantly better human computer interaction, where humans aren't interacting directly with the data in your preferred manner. It doesn’t explain why a randomized black box is needed. It only shows that we need better tools (i.e. not randomized and open).

deleted 3y ago

[deleted]

lukev 3y ago

I agree completely. Having LLMs use programming languages/tools/libraries built for humans is the wrong path to fully leverage their potential. They are capable of so much, but trying to use them as a drop-in replacement for humans isn't the best use for them. Frankly, it's impressive it works at all, but I think we will find hard limits on the actual complexity and reliability of the systems they can build in this manner. We still need intelligent, rigorous system design. LLMs have the strong potential to let system builders operate at a higher level of abstraction, but in my opinion having LLMs write Python or SQL is an evolutionary dead end. My own work is directly focused on this and what IS the proper place of a LLM in a data processing or knowledge system.