3 ms·

Maybe we shouldn't be selling products built on such a shaky foundation? Like Health Insurance products for example. [2]: https://insurtechdigital.com/articles

by tmpz22 2y ago

Maybe we shouldn't be selling products built on such a shaky foundation? Like Health Insurance products for example. [2]: https://insurtechdigital.com/articles/chatgpt-the-risks-and-rewards-for-insurers https://insurtechdigital.com/articles/chatgpt-the-risks-and-... --- please disregard [1] it was a terrible initial source I pulled of Google [1]: https://medium.com/artivatic/use-of-chatgpt-4-in-health-insurance-12548f3ae2c5 https://medium.com/artivatic/use-of-chatgpt-4-in-health-insu...

bcrl 2y ago
Minimum Viable Products are pretty much by definition built on shaky foundations. At least with software written by humans the failure modes are somewhat bounded by the architecture of the system as opposed to the who-knows-what-the-model-will-hallucinate of AI.
- nerdjon 2y ago
  I think that is the key problem, a traditional MVP is a mostly known entity. It may be missing some features, some bugs, etc. But it is an MVP not because it was necessarily rushed out the door (I mean... it was, but differently) but because it has some rough edges and is likely missing major features. Where what it seems we are getting with a lot of these companies shoving AI into something and calling it a product, is an MVP that is an MVP due to an unknown and untested nature.
  - mewpmewp2 2y ago
    But ultimately we have to test and release things to see what works and what doesn't. Very many usecases don't require perfect accuracy.
  - zerkten 2y ago
    The term MVP was cover for shoving poor quality software out on the market long before AI became involved. This is unfortunate, but inevitable when the term was popularized. AI is incredibly easy to tack on now, so people are doing that too.
    - nerdjon 2y ago
      That is true, but I think rushing to add AI features made it a completely different situation. We get a lot of MVP crap before, don't get me wrong. But at least it was understood crap. Sure it may have bugs in it and that is to be expected. But there was a limit in how wrong it could go. Since at the end of the day it was still limited to the code within the application and the server (if there is one). Meanwhile when an over-reliance on an LLM goes wrong, depending on how it goes wrong could be catastrophic. As we have seen time and time again just in the last couple months, when LLM's are shoved into something we seem to get a serious lack of testing under the guise of "beta".
Sharlin 2y ago
Building products on shaky foundations is a tried-and-true approach in IT business.
benreesman 2y ago
For a different point of view from someone with extremely credible credentials (learned this stuff from Hinton among many other things) and a much more sober and balanced take on all this I recommend the following interview with Nick Frosst (don’t be put off by the clickbait YouTube title, that’s a very silly caption): https://youtu.be/4JF1V2hzGKE https://youtu.be/4JF1V2hzGKE
SkyPuncher 2y ago
I’m not really sure this is an entirely fair argument. If you rely on third party packages of any type, you have dependencies that can rapidly and unexpectedly break with an update. Semantic versioning is supposed to help with this, but it doesn’t always help.

by tmpz22 2y ago

bcrl 2y ago

Minimum Viable Products are pretty much by definition built on shaky foundations. At least with software written by humans the failure modes are somewhat bounded by the architecture of the system as opposed to the who-knows-what-the-model-will-hallucinate of AI.

nerdjon 2y ago
I think that is the key problem, a traditional MVP is a mostly known entity. It may be missing some features, some bugs, etc. But it is an MVP not because it was necessarily rushed out the door (I mean... it was, but differently) but because it has some rough edges and is likely missing major features. Where what it seems we are getting with a lot of these companies shoving AI into something and calling it a product, is an MVP that is an MVP due to an unknown and untested nature.
- mewpmewp2 2y ago
  But ultimately we have to test and release things to see what works and what doesn't. Very many usecases don't require perfect accuracy.
- zerkten 2y ago
  The term MVP was cover for shoving poor quality software out on the market long before AI became involved. This is unfortunate, but inevitable when the term was popularized. AI is incredibly easy to tack on now, so people are doing that too.
  - nerdjon 2y ago
    That is true, but I think rushing to add AI features made it a completely different situation. We get a lot of MVP crap before, don't get me wrong. But at least it was understood crap. Sure it may have bugs in it and that is to be expected. But there was a limit in how wrong it could go. Since at the end of the day it was still limited to the code within the application and the server (if there is one). Meanwhile when an over-reliance on an LLM goes wrong, depending on how it goes wrong could be catastrophic. As we have seen time and time again just in the last couple months, when LLM's are shoved into something we seem to get a serious lack of testing under the guise of "beta".

Sharlin 2y ago

Building products on shaky foundations is a tried-and-true approach in IT business.

benreesman 2y ago

For a different point of view from someone with extremely credible credentials (learned this stuff from Hinton among many other things) and a much more sober and balanced take on all this I recommend the following interview with Nick Frosst (don’t be put off by the clickbait YouTube title, that’s a very silly caption): https://youtu.be/4JF1V2hzGKE https://youtu.be/4JF1V2hzGKE

SkyPuncher 2y ago

I’m not really sure this is an entirely fair argument. If you rely on third party packages of any type, you have dependencies that can rapidly and unexpectedly break with an update. Semantic versioning is supposed to help with this, but it doesn’t always help.