6 ms·

LLMs are a revolution in open source

70 points by Philpax 3y ago

aiman3 3y ago
the dependency of LLMs is very clear, without opensource, there is no LLMs, zero! not just LLMs, almost all AI/Machine-learning training are using linux, Slurm, python, i work in the HPC/ML field, include hardward/software/administration, i know this firsthand.
echelon 3y ago
LLMs are the death of open source. Labor might be free, but training resources seldom are. They won't be able to keep up.
- jrm4 3y ago
  That people actually entertain these ideas is SO WEIRD to me. The last sentence is just obviously false, precisely because the models are software that anyone can trade around. LoRA's, in other words. The fact that original models "might" -- and even that's a big might -- be hard to produce from scratch, so what? Operating systems are hard to produce from scratch, but anyone can grab Linux and do anything.
  - echelon 3y ago
    Your opinion doesn't make any sense to me. If it takes a hundred million dollars to train a SOTA LLM, who is going to pay for and put that in open source? Even Stability is starting to put restrictions on their releases (and is rumored to be searching for an acquirer).
    - jrm4 3y ago
      Two possibilities: 1) Why wouldn't Facebook (who looks like they're kind of doing that now) or anyone trying to compete with whoever the big monolith is threatening to be (which looks like OpenAI/Microsoft?) 2) Do they even need to, or is/can LLM development be incremental? I haven't heard anything about Google Fuchsia (sp?) lately either.
    - alerighi 3y ago
      The problem to me is another. Yes, there are people willingly to spend time and computing resources to train a model and improve it. The thing is, with software it's easy to verify (or, quite easy) that a contribution actually improves the code and thus gets accepted in the project. A modification to an AI model, for how models are made these day, it's completely opaque, a black box. How we can evaluate the modification to ensure that it actually improves the model and does not harm it, or worse introduce malicious behavior (since AI are also used to write code, for example train the AI to write malicious code). This is the real problem to solve, and to this day it's not solved. Probably the solution is to NOT have LARGE language model but rather have a multitude of small models (small enough that we can retrain them in hours with a normal PC) that are trained and tested individually, maintained by a group of persons in the open source community, and then merged together in a big model, just like a kernel is made up of thousands of individual modules that are then assembled together in a single software.
      - jrm4 3y ago
        But at this stage: why a big model AT ALL? Which is to say, a useful LLM is probably not going to be complex like an Operating System. (Now, an LLM that looks fancy and can fool dumb investors, sure, maybe -- but an actually useful one?)
- flohofwoe 3y ago
  They can only remix what already exists, not create new things, no?
  - jzombie 3y ago
    Technically, anything you could ever write in any [programmed or spoken] language is some sort of remix of existing tokens.
  - biofunsf 3y ago
    I think by most definitions LLMs can definitely create new things. As much as I can create new things by cobbling together open source projects and answers on stack overflow at least.
  - Der_Einzige 3y ago
    There are papers including one by Yann LeCunn which shows that the overwhelming majority of data found within high dimensional spaces is within decision boundries created by AI models. This implies that the overwhelming majority of things we think as humans are "novel" or "extrapolative" are in fact not novel and are nothing more than fancy interpolation. Basically, we humans cannot tell when something is "new to us" or "new to the universe" and it honestly hardly matters in practical reality. Finally, LLMs can totally "extrapolate" with techniques which encourage it to avoid its traditional training loss functions. A high temperature would do this, but it'll mostly just make your model crazy.
AlexanderDhoore 3y ago
This is kind of wishful thinking. I'm not saying you're wrong, but there's no real way to prove you're right. Sometimes open source wins, but not every time. The whole machine learning field is still too young to have a clear answer.
- drdg 3y ago
  Can you name some examples where open source won? In the past I wanted to believe this can be the future, where open source will somehow win (at least in some parts). What I see is that even the biggest projects are mere tools in the hands of the big corporations. Linux, Postgres, etc. All great! But have been assimilated. I cannot really consider them a win. It seems to me that it goes back and forth - it also seems to me that the advancements in LLMs will go a similar route.
  - toenail 3y ago
    > Linux, Postgres, etc. All great! But have been assimilated. I cannot really consider them a win. Are you trying to argue that free software becomes less free because specific groups of people contribute to it?
  - api 3y ago
    Open source has won hands down for developers. It’s basically a giant tool bin and parts yard for people who build things with software. It’s also useful to extremely tech savvy people who like to DIY homelab type stuff. In the consumer realm it has lost equally decisively. The reason, I think, is that the distance between software nerds can use and software the general public can (or wants) to use is significantly larger than anyone understood. Getting something to work gets it like 5% of the way to making it usable. Even worse, making it usable requires a huge amount of the kind of nit picky UI/UX work that programmers hate to do. This means they have to be paid to do it, which means usable software is many many times more expensive than technical software. The situation is hopeless unless people start paying for open software, which is hard because the FOSS movement taught everyone that software should be free.
    - 3y ago
KaiserPro 3y ago
The author currently misses the issue that all decent LLMs that have been released have been created by companies rather than opensource communities. Sure, a lot of models have been released under permissive licenses, but. Thats like releasing shareware. The special source for making _new_ LLMs comes from the dataset, and training clusters. None of which are cheap or easily run/financed by the community.
- ilaksh 3y ago
  Has anyone tried to take a distributed training framework and make it _really_ distributed?
- jrm4 3y ago
  Eh, I don't know; that's perhaps like giving credit to HP and Dell and perhaps Apple for Linux?
- cbarrick 3y ago
  More concretely, an ML mode is only "open source" if both the training code and the training data are freely available. Otherwise, it's not possible for the community to reproduce the model.
  - 3cats-in-a-coat 3y ago
    That's a bit like saying FOSS is open source only if a copy of the programmers is applied with the code. Everything that has a source, has another source that has produced that source. The algorithms behind creating LLMs are all published papers for all to read, the libraries (like TensorFlow) are themselves FOSS projects, and the data... is the open web for the most part. The Wikipedia dump alone is more than enough to get a very decent LLM shaped up. How an LLM is produced IS NO SECRET. It's just that to produce it you need millions (or for the more sophisticated ones: billions) in data center fees / power / GPU to train the model. So if the training scripts were included, you still can't make a LLama model yourself at home.
    - tomComb 3y ago
sva_ 3y ago
I have a feeling this article was partly written by a GPT. This part > For LLMs to be effective, they require a few things: And then that list are just a dead giveaway. I'm not nearly as cynical about GPT as many are (in fact I use them myself), but I think text written by it should be marked as such. edit: The author expressed that it was indeed not written using any GPT.
- dr_dshiv 3y ago
  > I think text written by it should be marked as such Good luck with that. It’s a tool.
  - flohofwoe 3y ago
    The problem is that it wastes the reader's time to parse the bullshit. Now of course a lot of humans write bullshit too, especially in areas that are at the peak of their current hype cycle ;)
- d-z-m 3y ago
  I think your comment was written by a GPT, does that make it true? Please refrain from making baseless accusations.
  - sva_ 3y ago
    Luckily I didn't make any baseless accusations.
    - Der_Einzige 3y ago
      Actually, you did make a totally baseless accusation and the fact that the person above you is being downvoted is evidence that the HN userbase is pretty awful at recognizing what "bases" are.
    - d-z-m 3y ago
      Speculation then. Just because someone has a bulleted list in their blog post doesn't mean it's AI generated.
willmadden 3y ago
I like the term surveillance capitalism. We need to move quickly, because the “moat” that big tech monopolies will seek to create will be regulatory, decelerationist, and authoritarian.
zlg_codes 3y ago
I still haven't found a use for LLMs. The propensity of them to get it wrong, and being fancy statistics machines based on existing data, just feels like a cheap trick. It's still a fancy Markov chain, it just has guard rails. There just isn't a place for them in my life.
- mrec 3y ago
  I'm not swigging the Kool-Aid yet either, but I think you're being a little dismissive. I've found ChatGPT useful for some sorts of questions that can only really be posed in natural language, particularly grammatical oddities that come up as I learn Spanish. Probably the single most impressive result I've had so far was with "What's that weird thing on the top of a P-38's engine nacelle that looks like a recessed sideways wheel?" Got it in one. I can't imagine getting anywhere with a conventional search engine there. I definitely wouldn't rely on it for anything professional, though.
  - naet 3y ago
    I copied your P38 question verbatim into Google and seem to have gotten the answer immediately, so not sure that's a big win for the LLM.
    - mrec 3y ago
      Huh. OK, I'm surprised, but that's a useful datapoint, thanks.
- jrm4 3y ago
  Like pretty much every other thing knowledge product, The only real value I see in them is in terms of helping humans think better. You can kind of tell them to "interview" or "question you" and they can spark your creativity.
- andrewmcwatters 3y ago
  If you look at the technical discussion from the perspective of individuals and companies creating sophisticated autocomplete, the whole fad seems ridiculous.
edwardjberry6 3y ago
[flagged]
- deleted 3y ago
  [deleted]
biofunsf 3y ago
> I build "brushes" for refactoring my code every day. Anyone know what the author means by brushes? Also any recommendations for non-Copilot IDE LLM integrations? I've tried a couple but they felt far behind Copilot in terms of quality and smooth IDE integration.
- notsylver 3y ago
  https://githubnext.com/projects/code-brushes/ https://githubnext.com/projects/code-brushes/ You select some code and apply a brush and it'll use a preset prompt to modify it: "Make Readable", "Add Types", "Document" are some of the options. Copilot has a pretty poor implementation, and mostly they just butcher your code - but they seem very powerful, especially with custom brushes.
- deleted 3y ago
  [deleted]
- larve 3y ago
  Things like "replace this PHP5 class with a new PHP8 readonly dataclass", "Convert this react components to have imageLinks as prop srcsets". Nothing fantastic, but damn this stuff is tedious to do manually. Now I can just batch process my files et voila.
alentred 3y ago
> LLMs are ultimately democratic LLMs reflect what they have been trained on, so the question is what they are trained on. In this line of thinking, at best they represent the groupthink, and at worst they are biased one way or another. Which is not to be taken negatively, but is something important to keep in mind, though.
- biofunsf 3y ago
  I don't think the author is saying the content LLMs produce is "democratic". They're just saying that they're democratic in the sense that anyone can run them and they have a strong open-source community behind them.
rightbyte 3y ago
I mean, I just the last hour setup the deepseek-coder-6.7B-instruct-GGUF model on my computer and tried it out with llama.cpp. It is maybe free ChatGTP level. If I could have run this when I was a student it would have been a real killer app. It is kinda impressive that I can run it on a crappy laptop. I some way this is also really scary, job wise. I would not recommend anyone getting into the programmer fields after the last years progress in these tools. But at least they will not be locked up to "OpenAI"'s servers ... I wonder if there could be a community effort doing model training, like the protein folding? If 30 000 computers ran some model training while not being used, would that be enough to get anywhere?
- trash_cat 3y ago
  Not getting into programming because anyone can use an LLM to do programming does not suddenly fill the productivity needs in an absolute sense. People can do more with less which also means we will quickly raise the demand for the new things we need to do.
  - Der_Einzige 3y ago
    That principle of demand increases up to the size of the supply makes sense until the supply becomes effectively infinity, which is what AI is doing. I'm not sure demand rises infinitely. I do believe that programming skills are heavily devalued in a world of effective AI, and most people on HN are a bit delusional about this fact, mainly because it's depressing to us computer touchers...
  - rightbyte 3y ago
    I think it will flip the tables completely. When "we" got into programming you more or less had to have aptitude for it and wade through dull documentation and still think it was fun to get anywhere. And there was a geek stigma. The recruiting body was really small for future programmers. LLM are so much help to get a newbie to some sort of entry level it gonna flood the market compared to when we entered. And the pay has got really good too, in itself a big enough problem leading to flooding from bellow.
- 3y ago
te_chris 3y ago
I also share their idea about reallocation of programming resource away from big corporations. There is so much bullshit empire building in our industry that serves only to waste talent. Small teams working on problems that need tech but can’t normally afford it should hopefully deliver something akin to a productivity shot in the arm across industries.
bilater 3y ago
The skepticism in the comments makes me even more bullish for this revolution.
unconed 3y ago
I treat usage of LLMs in coding as an admission of incompetence, and this post has only reinforced that view.
analyte123 3y ago
>I can't really see a future where big tech can keep up with a motivated community (and motivated it is, as any look at social media or tech websites will show you) As far as I can tell, many of these "motivated communities" organize on Discord (search is garbage, poor integrations, login-gated and invite-only in many cases), and also Twitter/X (now login-gated, API locked down). And beyond AI there are communities and lots of valuable content on Reddit (also locking down as hard as they practically can). I will be 0% surprised if Github locks down their platform, too. It's too soon to tell how much it will pay off, but almost as soon as ChatGPT came out, the obvious strategy of every company with user-generated content is for all the valuable communications, insights, and social graphs of their communities to be proprietary training data for the proprietary LLMs of the future.
deleted 3y ago
[deleted]

70 points by Philpax 3y ago

the dependency of LLMs is very clear, without opensource, there is no LLMs, zero! not just LLMs, almost all AI/Machine-learning training are using linux, Slurm, python, i work in the HPC/ML field, include hardward/software/administration, i know this firsthand.

echelon 3y ago

LLMs are the death of open source. Labor might be free, but training resources seldom are. They won't be able to keep up.

jrm4 3y ago
That people actually entertain these ideas is SO WEIRD to me. The last sentence is just obviously false, precisely because the models are software that anyone can trade around. LoRA's, in other words. The fact that original models "might" -- and even that's a big might -- be hard to produce from scratch, so what? Operating systems are hard to produce from scratch, but anyone can grab Linux and do anything.
- echelon 3y ago
  Your opinion doesn't make any sense to me. If it takes a hundred million dollars to train a SOTA LLM, who is going to pay for and put that in open source? Even Stability is starting to put restrictions on their releases (and is rumored to be searching for an acquirer).
  - jrm4 3y ago
    Two possibilities: 1) Why wouldn't Facebook (who looks like they're kind of doing that now) or anyone trying to compete with whoever the big monolith is threatening to be (which looks like OpenAI/Microsoft?) 2) Do they even need to, or is/can LLM development be incremental? I haven't heard anything about Google Fuchsia (sp?) lately either.
  - alerighi 3y ago
    The problem to me is another. Yes, there are people willingly to spend time and computing resources to train a model and improve it. The thing is, with software it's easy to verify (or, quite easy) that a contribution actually improves the code and thus gets accepted in the project. A modification to an AI model, for how models are made these day, it's completely opaque, a black box. How we can evaluate the modification to ensure that it actually improves the model and does not harm it, or worse introduce malicious behavior (since AI are also used to write code, for example train the AI to write malicious code). This is the real problem to solve, and to this day it's not solved. Probably the solution is to NOT have LARGE language model but rather have a multitude of small models (small enough that we can retrain them in hours with a normal PC) that are trained and tested individually, maintained by a group of persons in the open source community, and then merged together in a big model, just like a kernel is made up of thousands of individual modules that are then assembled together in a single software.
    - jrm4 3y ago
      But at this stage: why a big model AT ALL? Which is to say, a useful LLM is probably not going to be complex like an Operating System. (Now, an LLM that looks fancy and can fool dumb investors, sure, maybe -- but an actually useful one?)
flohofwoe 3y ago
They can only remix what already exists, not create new things, no?
- jzombie 3y ago
  Technically, anything you could ever write in any [programmed or spoken] language is some sort of remix of existing tokens.
- biofunsf 3y ago
  I think by most definitions LLMs can definitely create new things. As much as I can create new things by cobbling together open source projects and answers on stack overflow at least.
- Der_Einzige 3y ago
  There are papers including one by Yann LeCunn which shows that the overwhelming majority of data found within high dimensional spaces is within decision boundries created by AI models. This implies that the overwhelming majority of things we think as humans are "novel" or "extrapolative" are in fact not novel and are nothing more than fancy interpolation. Basically, we humans cannot tell when something is "new to us" or "new to the universe" and it honestly hardly matters in practical reality. Finally, LLMs can totally "extrapolate" with techniques which encourage it to avoid its traditional training loss functions. A high temperature would do this, but it'll mostly just make your model crazy.

AlexanderDhoore 3y ago

This is kind of wishful thinking. I'm not saying you're wrong, but there's no real way to prove you're right. Sometimes open source wins, but not every time. The whole machine learning field is still too young to have a clear answer.

drdg 3y ago
Can you name some examples where open source won? In the past I wanted to believe this can be the future, where open source will somehow win (at least in some parts). What I see is that even the biggest projects are mere tools in the hands of the big corporations. Linux, Postgres, etc. All great! But have been assimilated. I cannot really consider them a win. It seems to me that it goes back and forth - it also seems to me that the advancements in LLMs will go a similar route.
- toenail 3y ago
  > Linux, Postgres, etc. All great! But have been assimilated. I cannot really consider them a win. Are you trying to argue that free software becomes less free because specific groups of people contribute to it?
- api 3y ago
  Open source has won hands down for developers. It’s basically a giant tool bin and parts yard for people who build things with software. It’s also useful to extremely tech savvy people who like to DIY homelab type stuff. In the consumer realm it has lost equally decisively. The reason, I think, is that the distance between software nerds can use and software the general public can (or wants) to use is significantly larger than anyone understood. Getting something to work gets it like 5% of the way to making it usable. Even worse, making it usable requires a huge amount of the kind of nit picky UI/UX work that programmers hate to do. This means they have to be paid to do it, which means usable software is many many times more expensive than technical software. The situation is hopeless unless people start paying for open software, which is hard because the FOSS movement taught everyone that software should be free.
  - 3y ago

KaiserPro 3y ago

The author currently misses the issue that all decent LLMs that have been released have been created by companies rather than opensource communities. Sure, a lot of models have been released under permissive licenses, but. Thats like releasing shareware. The special source for making _new_ LLMs comes from the dataset, and training clusters. None of which are cheap or easily run/financed by the community.

ilaksh 3y ago
Has anyone tried to take a distributed training framework and make it _really_ distributed?
jrm4 3y ago
Eh, I don't know; that's perhaps like giving credit to HP and Dell and perhaps Apple for Linux?
cbarrick 3y ago
More concretely, an ML mode is only "open source" if both the training code and the training data are freely available. Otherwise, it's not possible for the community to reproduce the model.
- 3cats-in-a-coat 3y ago
  That's a bit like saying FOSS is open source only if a copy of the programmers is applied with the code. Everything that has a source, has another source that has produced that source. The algorithms behind creating LLMs are all published papers for all to read, the libraries (like TensorFlow) are themselves FOSS projects, and the data... is the open web for the most part. The Wikipedia dump alone is more than enough to get a very decent LLM shaped up. How an LLM is produced IS NO SECRET. It's just that to produce it you need millions (or for the more sophisticated ones: billions) in data center fees / power / GPU to train the model. So if the training scripts were included, you still can't make a LLama model yourself at home.
  - tomComb 3y ago

sva_ 3y ago

I have a feeling this article was partly written by a GPT. This part > For LLMs to be effective, they require a few things: And then that list are just a dead giveaway. I'm not nearly as cynical about GPT as many are (in fact I use them myself), but I think text written by it should be marked as such. edit: The author expressed that it was indeed not written using any GPT.

dr_dshiv 3y ago
> I think text written by it should be marked as such Good luck with that. It’s a tool.
- flohofwoe 3y ago
  The problem is that it wastes the reader's time to parse the bullshit. Now of course a lot of humans write bullshit too, especially in areas that are at the peak of their current hype cycle ;)
d-z-m 3y ago
I think your comment was written by a GPT, does that make it true? Please refrain from making baseless accusations.
- sva_ 3y ago
  Luckily I didn't make any baseless accusations.
  - Der_Einzige 3y ago
    Actually, you did make a totally baseless accusation and the fact that the person above you is being downvoted is evidence that the HN userbase is pretty awful at recognizing what "bases" are.
  - d-z-m 3y ago
    Speculation then. Just because someone has a bulleted list in their blog post doesn't mean it's AI generated.

willmadden 3y ago

I like the term surveillance capitalism. We need to move quickly, because the “moat” that big tech monopolies will seek to create will be regulatory, decelerationist, and authoritarian.

zlg_codes 3y ago

I still haven't found a use for LLMs. The propensity of them to get it wrong, and being fancy statistics machines based on existing data, just feels like a cheap trick. It's still a fancy Markov chain, it just has guard rails. There just isn't a place for them in my life.

mrec 3y ago
I'm not swigging the Kool-Aid yet either, but I think you're being a little dismissive. I've found ChatGPT useful for some sorts of questions that can only really be posed in natural language, particularly grammatical oddities that come up as I learn Spanish. Probably the single most impressive result I've had so far was with "What's that weird thing on the top of a P-38's engine nacelle that looks like a recessed sideways wheel?" Got it in one. I can't imagine getting anywhere with a conventional search engine there. I definitely wouldn't rely on it for anything professional, though.
- naet 3y ago
  I copied your P38 question verbatim into Google and seem to have gotten the answer immediately, so not sure that's a big win for the LLM.
  - mrec 3y ago
    Huh. OK, I'm surprised, but that's a useful datapoint, thanks.
jrm4 3y ago
Like pretty much every other thing knowledge product, The only real value I see in them is in terms of helping humans think better. You can kind of tell them to "interview" or "question you" and they can spark your creativity.
andrewmcwatters 3y ago
If you look at the technical discussion from the perspective of individuals and companies creating sophisticated autocomplete, the whole fad seems ridiculous.

edwardjberry6 3y ago

[flagged]

deleted 3y ago
[deleted]

biofunsf 3y ago

> I build "brushes" for refactoring my code every day. Anyone know what the author means by brushes? Also any recommendations for non-Copilot IDE LLM integrations? I've tried a couple but they felt far behind Copilot in terms of quality and smooth IDE integration.

notsylver 3y ago
https://githubnext.com/projects/code-brushes/ https://githubnext.com/projects/code-brushes/ You select some code and apply a brush and it'll use a preset prompt to modify it: "Make Readable", "Add Types", "Document" are some of the options. Copilot has a pretty poor implementation, and mostly they just butcher your code - but they seem very powerful, especially with custom brushes.
deleted 3y ago
[deleted]
larve 3y ago
Things like "replace this PHP5 class with a new PHP8 readonly dataclass", "Convert this react components to have imageLinks as prop srcsets". Nothing fantastic, but damn this stuff is tedious to do manually. Now I can just batch process my files et voila.

alentred 3y ago

> LLMs are ultimately democratic LLMs reflect what they have been trained on, so the question is what they are trained on. In this line of thinking, at best they represent the groupthink, and at worst they are biased one way or another. Which is not to be taken negatively, but is something important to keep in mind, though.

biofunsf 3y ago
I don't think the author is saying the content LLMs produce is "democratic". They're just saying that they're democratic in the sense that anyone can run them and they have a strong open-source community behind them.

rightbyte 3y ago

I mean, I just the last hour setup the deepseek-coder-6.7B-instruct-GGUF model on my computer and tried it out with llama.cpp. It is maybe free ChatGTP level. If I could have run this when I was a student it would have been a real killer app. It is kinda impressive that I can run it on a crappy laptop. I some way this is also really scary, job wise. I would not recommend anyone getting into the programmer fields after the last years progress in these tools. But at least they will not be locked up to "OpenAI"'s servers ... I wonder if there could be a community effort doing model training, like the protein folding? If 30 000 computers ran some model training while not being used, would that be enough to get anywhere?

trash_cat 3y ago
Not getting into programming because anyone can use an LLM to do programming does not suddenly fill the productivity needs in an absolute sense. People can do more with less which also means we will quickly raise the demand for the new things we need to do.
- Der_Einzige 3y ago
  That principle of demand increases up to the size of the supply makes sense until the supply becomes effectively infinity, which is what AI is doing. I'm not sure demand rises infinitely. I do believe that programming skills are heavily devalued in a world of effective AI, and most people on HN are a bit delusional about this fact, mainly because it's depressing to us computer touchers...
- rightbyte 3y ago
  I think it will flip the tables completely. When "we" got into programming you more or less had to have aptitude for it and wade through dull documentation and still think it was fun to get anywhere. And there was a geek stigma. The recruiting body was really small for future programmers. LLM are so much help to get a newbie to some sort of entry level it gonna flood the market compared to when we entered. And the pay has got really good too, in itself a big enough problem leading to flooding from bellow.
3y ago

te_chris 3y ago

I also share their idea about reallocation of programming resource away from big corporations. There is so much bullshit empire building in our industry that serves only to waste talent. Small teams working on problems that need tech but can’t normally afford it should hopefully deliver something akin to a productivity shot in the arm across industries.

bilater 3y ago

The skepticism in the comments makes me even more bullish for this revolution.

unconed 3y ago

I treat usage of LLMs in coding as an admission of incompetence, and this post has only reinforced that view.

analyte123 3y ago

>I can't really see a future where big tech can keep up with a motivated community (and motivated it is, as any look at social media or tech websites will show you) As far as I can tell, many of these "motivated communities" organize on Discord (search is garbage, poor integrations, login-gated and invite-only in many cases), and also Twitter/X (now login-gated, API locked down). And beyond AI there are communities and lots of valuable content on Reddit (also locking down as hard as they practically can). I will be 0% surprised if Github locks down their platform, too. It's too soon to tell how much it will pay off, but almost as soon as ChatGPT came out, the obvious strategy of every company with user-generated content is for all the valuable communications, insights, and social graphs of their communities to be proprietary training data for the proprietary LLMs of the future.

deleted 3y ago

[deleted]