6 ms·

> If you want to gatekeep or only create stuff to boost your own ego or portfolio, then AI might be an issue, if you actually want to build stuff, AI is godsend

> If you want to gatekeep or only create stuff to boost your own ego or portfolio, then AI might be an issue, if you actually want to build stuff, AI is godsend. Exactly this. I've been very confused by the free software advocates that seemed to hate AI until I realized their reasons for releasing software under an open source license were very different than what I assumed they were.

za_creature 8d ago
While Stallman may have been originally upset about his printer, the primary motivation behind open source software is: Nothing up my sleeve. This of course requires audit and that requires that the code is written for human consumption, otherwise nobody will bother. Sure you can vibe a printer driver but how sure are you that your LLM didn't include a backdoor in the millions line of slop?
- williamcotton 8d ago
  How sure are you that any printer driver didn’t include a backdoor? What’s the difference between the LLM code with the backdoor and the human code with the backdoor?
  - za_creature 8d ago
    The ability to audit the code. If not that, the ability to trust the reputation of the author which creates an incentive not to willingly insert a backdoor. Yes, `npm install` was always bullshit because most people didn't bother to check, which is exactly why it was exploited multiple times, which created a conversation about "supply chain security". If you want to argue that "npm changed the world" then you are correct. It did not change the world for the better though.
    - williamcotton 8d ago
      Why is LLM code always harder to audit? So LLMs might decide to add backdoors without prompting?
      - za_creature 8d ago
        I'm less worried about what LLMs decide and more about what their system prompts tell them to do.
- rpdillon 8d ago
  Trusting software produced by others is not a new problem; I consider AI to be a tool, and so my techniques for establishing trust are the same as they have always been. I look at the person wielding the tool.
  - za_creature 8d ago
    Then you best hope that person has read https://people.cs.umass.edu/~emery/classes/cmpsci691st/readings/Sec/Reflections-on-Trusting-Trust.pdf https://people.cs.umass.edu/~emery/classes/cmpsci691st/readi...
  - ben_w 8d ago
    > I look at the person wielding the tool. I look at the whole chain of influence behind them, and it's quite a lot more upsetting than the smiling face I interact with. The whole "AI is a Lovecraftian tentacle monster wearing a smiley face" thing applies to simple bureaucracies (replace Lovecraft with Kafka), to corporations (replace Lovecraft with IDK, most anarchists?), and governments (Orwell?) Tools made by tools made by tools, along more steps than most people know even when their job is one of them. Somewhere there's a kid working a dangerous mine without the right safety equipment, elsewhere there's a sweatshop, another place a "reeducation camp". But who do I see? A cashier, mostly. Someone whose job involves smiling to customers even when we're idiots.
    - rpdillon 8d ago
      I'm not sure what point you're trying to make. I put trust in individual people, not organizations. You seem to be making a point about supply chains, but I'm not sure how it relates to the point about trust. You seem to be making a different point about some kind of exploitation.
      - ben_w 8d ago
        Almost everything I interact with was made by an organisation (or a disorganisation), not by an individual. Say I download an app. Who made it? The programmer? Their PM? Apple's store requirements? The US government, for whom there was a special tickbox I had to agree to last time I uploaded an app? Trust is for the mechanic, driver, builder; but for 90% of my interactions I have to trust my government set good rules and other people followed them. I can't do this with AI, neither good rules nor them being followed, but I also can't do it when the OS company and app devs are foreign, as they generally are to me now.
- mitxela 8d ago
  That's a motivation for source available. You can have commercial copyrighted software that's NUMS. You can deliver the customer source code (slightly customized for traitor-telling of course) and tell them to do what they want but never share it. grsecurity even managed this with GPL software!
wiz21c 8d ago
I release my code as free software for two reasons: 1. The credits for my code are protected (with the GPL you have to tell where your code originates from). That's the ego part. It's important to me because I'm not paid for my software. So credits are an important reward. 2. I want people to think twice about reusing my code. I use the GPL license because I think sharing software is the ultimate goal. So I force people to share my software by using the GPL. That may sound "extreme" (that's the whole open source vs free software debate) but, not being a full time politician, I can't change laws to push society in the direction I want. At my level, that "push" is the GPL choice. Maybe it's not noble enough, maybe it's cowardice, but it's my way (compare that with those who simply don't care). AI severely weakens both of these. And for people like me, this forces us to reconsider our position. For my part, I accept the legal point of view that A.I. doesn't steal code, and just reproduces the ideas in the code. So, as far as ideas can flow in society, I'm OK with that (that's the principle behind copyright laws). If AI has its way, one day one will not need to write software, we'll just ask the AI. In that case, software will be dead and free software will die with it too. By then I'll do my "local politics" another way and follow the next RMS.
- za_creature 8d ago
  > A.I. doesn't steal code, and just reproduces the ideas in the code Then they don't need to train on github, no? Why not release a new model trained from Knuth's Art of Programming, Cormen's Introduction to Algorithms and the C specification. Feel free to throw in any other published literature related to STEM, but stick to the code samples from the books. I'm certain it'll be able to change the color of a CSS button, right?
  - kolinko 8d ago
    What you said doesn't disagree with what the parent said. LLM can be trained on a code and at the same time reproduce the core ideas. That's what LLMs do after all - they convert the training data into their own internal models and representations, and then reproduce the ideas. Sure, some things/patterns, that were repeated multiple times, LLMs will tend to repeat verbatim as well, but that's not that big of a problem. As a person who invented a few algorithms on my own I absolutely love LLMs and I don't mind them being trained on my work, but yeah - I've been way less likely to publish open source over the last year. In the past, if some of my stuff got traction, the credit was close to automatic (early adopters credited or at least knew where they got it from). Nowadays, LLMs will train on these ideas, rewrite them, and give no credit. Still, I prefer this to having no LLMs at all. > but stick to the code samples from the books. > I'm certain it'll be able to change the color of a CSS button, right? A good enough LLM will just decompile a browser, figure out CSS spec from it, and yes - figure out how to change the color of a CSS button from first principles. There is no point to do this with CSS, but with other things it's now easier to just dig through sorces or direct bytecode than to bother checking docs.
matsemann 8d ago
Exactly not this. Way to straw man all counter arguments before they even got any.
- rpdillon 8d ago
  I haven't heard any coherent arguments other than getting credit and building a portfolio. Are there others?
  - watwut 8d ago
    Neither of your guesses is reality based. Open source contributors get no credit in general. To the point that even commenters on HN dont seem to know who they, in general are. That being said there is absolutely nothing wrong with building a portfolio. It is 100% valid motivation. The only issue with that theory is that no one cares about your open source contributions. Not employers, not peers, no one. What I do find fascinating tho is that fans of the most selfish egoistical companies and tech groups ... insinuate artists, open source programmers, writers or anyone else is selfish or otherwise disapponting if any part of their work involve actual human motivation.
    - rpdillon 8d ago
      I wouldn't say that wanting to contribute to a shared commons of cool stuff isn't a human motivation. And I'm not deriding those that build open source to boost their own reputation. I'm just much less interested in their work than the work of folks that are motivated by improving the commons (even if they get no credit).
      - watwut 8d ago
        You are imagining people in your head. Most serious open source contributions are paid for. Per FSF survey overwhelmingly so. Because, they are a lot of work. They are not hobbies or kitchen soup two hours a week effort. These require a lot of sustained effort and time, that would either takes time from your paid job, or would make you effectively work 60 hours a week which means you have no other life. End result is that these are paid positions, so.
intended 8d ago
Drive by code dumps from people who have "democratized Natural Language code" which pushes all the work of verifying whether it works on the few active maintainers is hell. Any collective group with open contribution will end up hating it. Why is this a surprise?

za_creature 8d ago

While Stallman may have been originally upset about his printer, the primary motivation behind open source software is: Nothing up my sleeve. This of course requires audit and that requires that the code is written for human consumption, otherwise nobody will bother. Sure you can vibe a printer driver but how sure are you that your LLM didn't include a backdoor in the millions line of slop?

williamcotton 8d ago
How sure are you that any printer driver didn’t include a backdoor? What’s the difference between the LLM code with the backdoor and the human code with the backdoor?
- za_creature 8d ago
  The ability to audit the code. If not that, the ability to trust the reputation of the author which creates an incentive not to willingly insert a backdoor. Yes, `npm install` was always bullshit because most people didn't bother to check, which is exactly why it was exploited multiple times, which created a conversation about "supply chain security". If you want to argue that "npm changed the world" then you are correct. It did not change the world for the better though.
  - williamcotton 8d ago
    Why is LLM code always harder to audit? So LLMs might decide to add backdoors without prompting?
    - za_creature 8d ago
      I'm less worried about what LLMs decide and more about what their system prompts tell them to do.
rpdillon 8d ago
Trusting software produced by others is not a new problem; I consider AI to be a tool, and so my techniques for establishing trust are the same as they have always been. I look at the person wielding the tool.
- za_creature 8d ago
  Then you best hope that person has read https://people.cs.umass.edu/~emery/classes/cmpsci691st/readings/Sec/Reflections-on-Trusting-Trust.pdf https://people.cs.umass.edu/~emery/classes/cmpsci691st/readi...
- ben_w 8d ago
  > I look at the person wielding the tool. I look at the whole chain of influence behind them, and it's quite a lot more upsetting than the smiling face I interact with. The whole "AI is a Lovecraftian tentacle monster wearing a smiley face" thing applies to simple bureaucracies (replace Lovecraft with Kafka), to corporations (replace Lovecraft with IDK, most anarchists?), and governments (Orwell?) Tools made by tools made by tools, along more steps than most people know even when their job is one of them. Somewhere there's a kid working a dangerous mine without the right safety equipment, elsewhere there's a sweatshop, another place a "reeducation camp". But who do I see? A cashier, mostly. Someone whose job involves smiling to customers even when we're idiots.
  - rpdillon 8d ago
    I'm not sure what point you're trying to make. I put trust in individual people, not organizations. You seem to be making a point about supply chains, but I'm not sure how it relates to the point about trust. You seem to be making a different point about some kind of exploitation.
    - ben_w 8d ago
      Almost everything I interact with was made by an organisation (or a disorganisation), not by an individual. Say I download an app. Who made it? The programmer? Their PM? Apple's store requirements? The US government, for whom there was a special tickbox I had to agree to last time I uploaded an app? Trust is for the mechanic, driver, builder; but for 90% of my interactions I have to trust my government set good rules and other people followed them. I can't do this with AI, neither good rules nor them being followed, but I also can't do it when the OS company and app devs are foreign, as they generally are to me now.
mitxela 8d ago
That's a motivation for source available. You can have commercial copyrighted software that's NUMS. You can deliver the customer source code (slightly customized for traitor-telling of course) and tell them to do what they want but never share it. grsecurity even managed this with GPL software!

wiz21c 8d ago

I release my code as free software for two reasons: 1. The credits for my code are protected (with the GPL you have to tell where your code originates from). That's the ego part. It's important to me because I'm not paid for my software. So credits are an important reward. 2. I want people to think twice about reusing my code. I use the GPL license because I think sharing software is the ultimate goal. So I force people to share my software by using the GPL. That may sound "extreme" (that's the whole open source vs free software debate) but, not being a full time politician, I can't change laws to push society in the direction I want. At my level, that "push" is the GPL choice. Maybe it's not noble enough, maybe it's cowardice, but it's my way (compare that with those who simply don't care). AI severely weakens both of these. And for people like me, this forces us to reconsider our position. For my part, I accept the legal point of view that A.I. doesn't steal code, and just reproduces the ideas in the code. So, as far as ideas can flow in society, I'm OK with that (that's the principle behind copyright laws). If AI has its way, one day one will not need to write software, we'll just ask the AI. In that case, software will be dead and free software will die with it too. By then I'll do my "local politics" another way and follow the next RMS.

za_creature 8d ago
> A.I. doesn't steal code, and just reproduces the ideas in the code Then they don't need to train on github, no? Why not release a new model trained from Knuth's Art of Programming, Cormen's Introduction to Algorithms and the C specification. Feel free to throw in any other published literature related to STEM, but stick to the code samples from the books. I'm certain it'll be able to change the color of a CSS button, right?
- kolinko 8d ago
  What you said doesn't disagree with what the parent said. LLM can be trained on a code and at the same time reproduce the core ideas. That's what LLMs do after all - they convert the training data into their own internal models and representations, and then reproduce the ideas. Sure, some things/patterns, that were repeated multiple times, LLMs will tend to repeat verbatim as well, but that's not that big of a problem. As a person who invented a few algorithms on my own I absolutely love LLMs and I don't mind them being trained on my work, but yeah - I've been way less likely to publish open source over the last year. In the past, if some of my stuff got traction, the credit was close to automatic (early adopters credited or at least knew where they got it from). Nowadays, LLMs will train on these ideas, rewrite them, and give no credit. Still, I prefer this to having no LLMs at all. > but stick to the code samples from the books. > I'm certain it'll be able to change the color of a CSS button, right? A good enough LLM will just decompile a browser, figure out CSS spec from it, and yes - figure out how to change the color of a CSS button from first principles. There is no point to do this with CSS, but with other things it's now easier to just dig through sorces or direct bytecode than to bother checking docs.

matsemann 8d ago

Exactly not this. Way to straw man all counter arguments before they even got any.

rpdillon 8d ago
I haven't heard any coherent arguments other than getting credit and building a portfolio. Are there others?
- watwut 8d ago
  Neither of your guesses is reality based. Open source contributors get no credit in general. To the point that even commenters on HN dont seem to know who they, in general are. That being said there is absolutely nothing wrong with building a portfolio. It is 100% valid motivation. The only issue with that theory is that no one cares about your open source contributions. Not employers, not peers, no one. What I do find fascinating tho is that fans of the most selfish egoistical companies and tech groups ... insinuate artists, open source programmers, writers or anyone else is selfish or otherwise disapponting if any part of their work involve actual human motivation.
  - rpdillon 8d ago
    I wouldn't say that wanting to contribute to a shared commons of cool stuff isn't a human motivation. And I'm not deriding those that build open source to boost their own reputation. I'm just much less interested in their work than the work of folks that are motivated by improving the commons (even if they get no credit).
    - watwut 8d ago
      You are imagining people in your head. Most serious open source contributions are paid for. Per FSF survey overwhelmingly so. Because, they are a lot of work. They are not hobbies or kitchen soup two hours a week effort. These require a lot of sustained effort and time, that would either takes time from your paid job, or would make you effectively work 60 hours a week which means you have no other life. End result is that these are paid positions, so.

intended 8d ago

Drive by code dumps from people who have "democratized Natural Language code" which pushes all the work of verifying whether it works on the few active maintainers is hell. Any collective group with open contribution will end up hating it. Why is this a surprise?