9 ms·

I'm actually worried of the inevitable enshitification of Open AI. Right now, GPT 4 is great and I use it every day. I rely less and less on Google and save a

by keyle 3y ago

I'm actually worried of the inevitable enshitification of Open AI. Right now, GPT 4 is great and I use it every day. I rely less and less on Google and save a ton of time. Google has gone downhill hard for a few years, thanks to all these silly blog engines like medium, substack and the like. I'm really worried and would like to 'freeze' GPT 4 as is, at least with the data up to Sep 2021, and for them to not change or tweak any of it further. I wish we could have an offline solution just as good, or a guarantee of GPT 4 LTS. I don't mind paying so as long as it's useful. I'm convinced that inevitably, every successful engineering driven business falls into a marketing business; and once market research and sales drive every business decision, the product is boned.

version_five 3y ago
This is a good point and I'm surprised it hasn't had more attention. We've seen history repeat itself enough times that it's inevitable. Probably the best you can do is download a few of the full precision 70B or bigger models (LLaMA etc) to at least have some state of the art weights around you could use yourself if you wanted if/when openAI goes to shit. The nice thing about the language models is as least they're relatively compact, compared to google search which you'd never be able to host locally. Though thinking about it, in a way LLMs are local copies of the "good" internet. The big ones mostly seem to know what information there is to know and have none of the crap and link/content farms that is now most of the internet.
- MPSimmons 3y ago
  >... compared to google search which you'd never be able to host locally That's something I haven't thought about for a long time. How big is the Google index, would you say? Petabytes or exabytes, or am I vastly overestimating?
  - raincole 3y ago
    > over 100,000,000 gigabytes Accodrding to Google: https://www.google.com/search/howsearchworks/how-search-works/organizing-information https://www.google.com/search/howsearchworks/how-search-work...
  - hansvm 3y ago
    Probably petabytes for the text index and exabytes for the whole index (including images and video). The entire textual internet is hundreds of terabytes uncompressed, and indexes tend to balloon storage size to make serving tailored queries faster and cheaper. That idea shifts a bit with images/video where vector/ML techniques dominate; you get a reduction in quality, but quality is good enough (mostly, probably, we think), and you have within a small constant multiple one way or the other of the underlying dataset size for the index size (as opposed to text, where 100x-1000x one way or another isn't uncommon). If the index is less than petabytes then it's impossible to serve many long tail queries even just based on text content. IME those queries _have_ gotten progressively worse results the last few years, so maybe the techniques have changed, but when the money printer is "search" and a petabyte of data is a fraction of a full-time engineer in cost to store, I doubt they would have cut costs that aggressively.
  - jdshaffer 3y ago
    That reminds me of the time my friend downloaded the ENTIRE Yahoo! search database to a 1.44MB floppy. I think that was back in 1993. laugh
satvikpendem 3y ago
That's why I think open source LLMs are the ones that will actually persist throughout the commercial enshittification and reign supreme at the end.
- keyle 3y ago
  I'm with you but I'm yet to find a solution that isn't both rocket science and requiring rocket hardware.
  - satvikpendem 3y ago
    What do you mean?
  - selfhoster11 3y ago
    Not in my experience. For an ML hobbyist, I don't have particularly strong hardware - around 40 GB of system RAM and a very small GPU with 8GB of RAM. I can run all sizes of models, up to and including 70B (albeit slowly). 13 is definitely the most usable on this particular hardware, given the trade-off between model capabilities and generation speed.
- Someone1234 3y ago
  I'm more interested in Open Source/Local LLM's ability to consume private data as an internal knowledgeable. For example, feed in your entire source repo(s) and then ask it questions about where something is or how it works.
  - zurtri 3y ago
    I am constantly searching for this. To point a locally hosting LLM to a locally hosted code repo - and then start improving it from there would be awesome.
    - pbmonster 3y ago
      Depending on how serious you are about this (and how well you can slap together a few different python packages), it is very doable today. Get one of the better llama versions fine-tuned on code (e.g. WizardCoder), take your entire code base and create embeddings from it, put those into a vector database. Now, every time you ask your LLM a question about your code base, you first turn that prompt into an embedding and perform a search on your vector database. The results of that search are appended as context to the actual prompt before passing it to the LLM itself. There's tons of packages that help with all of that, Langchain and Faiss are probably the most popular right now.
Zetice 3y ago
Have you tried Googling something factual lately? Bard just gives you the answer.
- HWR_14 3y ago
  I just tried. I did not get a factual answer from bard.
  - Zetice 3y ago
    Huh, I did. Maybe it only works for some people!
    - ATMLOTTOBEER 3y ago
      He is determined to see no value. Today I couldn’t find the mouse acceleration toggle in macOS, and google turned up a bunch of mediocre results I would have to sift through. ChatGPT just gave me the terminal command I was looking for immediately.
      - HWR_14 3y ago
        Or, I literally asked a factual question of Google (several times, although similar questions) and didn't get a bard generated response. Strange to read "Google searches aren't doing this for me" as "they are doing it for me and I don't like it"
        Kiro 3y ago
        So what was the question?
        WhrRTheBaboons 3y ago
        Bard isnt available in EU (for example) No question you ask will return an answer
        3y ago
Gigachad 3y ago
I'm almost hoping for some enshittification. It's so depressing seeing how many skills and jobs have been instantly devalued. Take a look at the artist communities and it's total depression that peoples life passion and source of income has just been deleted over the course of a few months. We aren't talking about burger flipping and checkout scanners. This is a creative pursuit which takes years of grinding to become proficient, just wiped out. Right now the current models and tech aren't quite enough to replace real humans, but the rate of progression feels like they will soon. If it all just halted here for a while, that would be quite comforting for a lot of people.
- kaashif 3y ago
  If things slow down a bit then speed up again in a few years once economics improve or whatever, does that really make any difference in the long run? What's the long term solution here? We're not even talking about decades here, probably.
  - Gigachad 3y ago
    Part of me hopes we have hit a kind of wall where we get what we have now, maybe with some more polished UI, but it doesn't continue the exponential improvement path it's on. If it all slowed down, people would have plenty of time to readjust. Rather than just instantly deleting peoples careers. Human artists can quite easily beat the current tech, but there is no real indication that the current stuff is the best we can juice out of the current methods. Which is the depressing part. Why even bother to learn anything when AI will obsolete your effort before you finish.
- ToucanLoucan 3y ago
  Not even deleted! Actively stolen. No one has yet grappled with the reality that especially in the realm of generative art, these models do not work without a basis of absolutely stressful amounts of stolen artwork to train them. They simply do. Not. Exist. Without that baseline unethical contribution that tons of artists made without their knowledge and certainly without their consent. None of the major figures behind any of this tech will acknowledge this. They make some hand-wavy non-committal gestures towards fair use and "how artists learn" and act like that's an answer, completely discarding tons of points not worth revisiting here because it's frankly been beaten to death already and if you're still fence sitting or pro generative art at this point, it's because you want/need to be to prevent cognitive dissonance in one way or another. And to be clear: this stuff is SO COOL. The idea of entering prompts and getting at least vaguely unique and interesting output is incredibly cool. The notion of machines that learn, even in a restrictive way as the current models do, is so fascinating. And of course, like any cool and interesting new tech, it was immediately adopted/hijacked by the worst people imaginable, determined to fill Amazon self-publishing with generated hack kids books filled with generated text and generated art, at the lowest cost point possible, sold on the cheap, with the idea that they could make them convincing enough to trick some poor overworked parents into buying them and pocketing the cash. Just grift. Every new technology we get now, as few as the true new innovations they are, are always, always, immediately, co-opted by the worst actors in the space and turned to utter garbage.
tayo42 3y ago
Maybe I'm just falling for the marketing, but isn't a big part of open ai that it's not profit drive? All companies collapse but it seems structured to not be in the short term
aceazzameen 3y ago
I too hope Open AI will always have a version with frozen data up to Sept 2021. At the very least for historical purposes. I know the data will start to go stale and not have the newest most relevant information. But data after that date will start mixing with content created with the original data, making it more unreliable over time.

by keyle 3y ago

version_five 3y ago

This is a good point and I'm surprised it hasn't had more attention. We've seen history repeat itself enough times that it's inevitable. Probably the best you can do is download a few of the full precision 70B or bigger models (LLaMA etc) to at least have some state of the art weights around you could use yourself if you wanted if/when openAI goes to shit. The nice thing about the language models is as least they're relatively compact, compared to google search which you'd never be able to host locally. Though thinking about it, in a way LLMs are local copies of the "good" internet. The big ones mostly seem to know what information there is to know and have none of the crap and link/content farms that is now most of the internet.

MPSimmons 3y ago
>... compared to google search which you'd never be able to host locally That's something I haven't thought about for a long time. How big is the Google index, would you say? Petabytes or exabytes, or am I vastly overestimating?
- raincole 3y ago
  > over 100,000,000 gigabytes Accodrding to Google: https://www.google.com/search/howsearchworks/how-search-works/organizing-information https://www.google.com/search/howsearchworks/how-search-work...
- hansvm 3y ago
  Probably petabytes for the text index and exabytes for the whole index (including images and video). The entire textual internet is hundreds of terabytes uncompressed, and indexes tend to balloon storage size to make serving tailored queries faster and cheaper. That idea shifts a bit with images/video where vector/ML techniques dominate; you get a reduction in quality, but quality is good enough (mostly, probably, we think), and you have within a small constant multiple one way or the other of the underlying dataset size for the index size (as opposed to text, where 100x-1000x one way or another isn't uncommon). If the index is less than petabytes then it's impossible to serve many long tail queries even just based on text content. IME those queries _have_ gotten progressively worse results the last few years, so maybe the techniques have changed, but when the money printer is "search" and a petabyte of data is a fraction of a full-time engineer in cost to store, I doubt they would have cut costs that aggressively.
- jdshaffer 3y ago
  That reminds me of the time my friend downloaded the ENTIRE Yahoo! search database to a 1.44MB floppy. I think that was back in 1993. laugh

satvikpendem 3y ago

That's why I think open source LLMs are the ones that will actually persist throughout the commercial enshittification and reign supreme at the end.

keyle 3y ago
I'm with you but I'm yet to find a solution that isn't both rocket science and requiring rocket hardware.
- satvikpendem 3y ago
  What do you mean?
- selfhoster11 3y ago
  Not in my experience. For an ML hobbyist, I don't have particularly strong hardware - around 40 GB of system RAM and a very small GPU with 8GB of RAM. I can run all sizes of models, up to and including 70B (albeit slowly). 13 is definitely the most usable on this particular hardware, given the trade-off between model capabilities and generation speed.
Someone1234 3y ago
I'm more interested in Open Source/Local LLM's ability to consume private data as an internal knowledgeable. For example, feed in your entire source repo(s) and then ask it questions about where something is or how it works.
- zurtri 3y ago
  I am constantly searching for this. To point a locally hosting LLM to a locally hosted code repo - and then start improving it from there would be awesome.
  - pbmonster 3y ago
    Depending on how serious you are about this (and how well you can slap together a few different python packages), it is very doable today. Get one of the better llama versions fine-tuned on code (e.g. WizardCoder), take your entire code base and create embeddings from it, put those into a vector database. Now, every time you ask your LLM a question about your code base, you first turn that prompt into an embedding and perform a search on your vector database. The results of that search are appended as context to the actual prompt before passing it to the LLM itself. There's tons of packages that help with all of that, Langchain and Faiss are probably the most popular right now.

Zetice 3y ago

Have you tried Googling something factual lately? Bard just gives you the answer.

HWR_14 3y ago
I just tried. I did not get a factual answer from bard.
- Zetice 3y ago
  Huh, I did. Maybe it only works for some people!
  - ATMLOTTOBEER 3y ago
    He is determined to see no value. Today I couldn’t find the mouse acceleration toggle in macOS, and google turned up a bunch of mediocre results I would have to sift through. ChatGPT just gave me the terminal command I was looking for immediately.
    - HWR_14 3y ago
      Or, I literally asked a factual question of Google (several times, although similar questions) and didn't get a bard generated response. Strange to read "Google searches aren't doing this for me" as "they are doing it for me and I don't like it"
      - Kiro 3y ago
        So what was the question?
        WhrRTheBaboons 3y ago
        Bard isnt available in EU (for example) No question you ask will return an answer
      - 3y ago

Gigachad 3y ago

I'm almost hoping for some enshittification. It's so depressing seeing how many skills and jobs have been instantly devalued. Take a look at the artist communities and it's total depression that peoples life passion and source of income has just been deleted over the course of a few months. We aren't talking about burger flipping and checkout scanners. This is a creative pursuit which takes years of grinding to become proficient, just wiped out. Right now the current models and tech aren't quite enough to replace real humans, but the rate of progression feels like they will soon. If it all just halted here for a while, that would be quite comforting for a lot of people.

kaashif 3y ago
If things slow down a bit then speed up again in a few years once economics improve or whatever, does that really make any difference in the long run? What's the long term solution here? We're not even talking about decades here, probably.
- Gigachad 3y ago
  Part of me hopes we have hit a kind of wall where we get what we have now, maybe with some more polished UI, but it doesn't continue the exponential improvement path it's on. If it all slowed down, people would have plenty of time to readjust. Rather than just instantly deleting peoples careers. Human artists can quite easily beat the current tech, but there is no real indication that the current stuff is the best we can juice out of the current methods. Which is the depressing part. Why even bother to learn anything when AI will obsolete your effort before you finish.
ToucanLoucan 3y ago
Not even deleted! Actively stolen. No one has yet grappled with the reality that especially in the realm of generative art, these models do not work without a basis of absolutely stressful amounts of stolen artwork to train them. They simply do. Not. Exist. Without that baseline unethical contribution that tons of artists made without their knowledge and certainly without their consent. None of the major figures behind any of this tech will acknowledge this. They make some hand-wavy non-committal gestures towards fair use and "how artists learn" and act like that's an answer, completely discarding tons of points not worth revisiting here because it's frankly been beaten to death already and if you're still fence sitting or pro generative art at this point, it's because you want/need to be to prevent cognitive dissonance in one way or another. And to be clear: this stuff is SO COOL. The idea of entering prompts and getting at least vaguely unique and interesting output is incredibly cool. The notion of machines that learn, even in a restrictive way as the current models do, is so fascinating. And of course, like any cool and interesting new tech, it was immediately adopted/hijacked by the worst people imaginable, determined to fill Amazon self-publishing with generated hack kids books filled with generated text and generated art, at the lowest cost point possible, sold on the cheap, with the idea that they could make them convincing enough to trick some poor overworked parents into buying them and pocketing the cash. Just grift. Every new technology we get now, as few as the true new innovations they are, are always, always, immediately, co-opted by the worst actors in the space and turned to utter garbage.

tayo42 3y ago

Maybe I'm just falling for the marketing, but isn't a big part of open ai that it's not profit drive? All companies collapse but it seems structured to not be in the short term

aceazzameen 3y ago

I too hope Open AI will always have a version with frozen data up to Sept 2021. At the very least for historical purposes. I know the data will start to go stale and not have the newest most relevant information. But data after that date will start mixing with content created with the original data, making it more unreliable over time.