4 ms·

I think if you specify a technology in your prompt, any LLM should use that technology in its response. If you don't specify a technology, and that is an import

I think if you specify a technology in your prompt, any LLM should use that technology in its response. If you don't specify a technology, and that is an important consideration in the answer, it should clarify and ask about technology choices, and if you don't know, it can make a recommendation. LLMs should not have hard-wired preferences through providers' prompt structure. And while LLMs are stochastic parrots, and are likely to infer React if a lot of the training corpus mentions React, work should be done to actively prevent biases like this. If we can't get this right with JS frameworks, how are we going to solve it for more nuanced structural biases around ethnicity, gender, religion or political perspective? What I'm most concerned about here is that Anthropic is taking investment from tech firms who vendor dev tooling - it would not take much for them to "prefer" one of those proprietary toolchains. We might not have much of a problem with React today, but what if your choice of LLM started to determine if you could or couldn't get recommendations on AWS vs Azure vs GCP vs bare metal/roll your own? Or if it suggested only commercial tools instead of F/LOSS? And to take that to its logical conclusion, if that's happening, how do I know that the history assignment a kid is asking for help with isn't sneaking in an extreme viewpoint - and I don't care if it's extreme left or right, just warped by a political philosophy to be disconnected from truth - that the kid just accepts as truth?

avbanks 2y ago
This is actually a very interesting insight, not only do you have to worry about sponsored results but people could game the system by spamming their library/language in a places which will be included in the training set of models. This will also present a significant challenge for security, because I can have a malicious library/package spam it in paths that will be picked up in the training set and have that package be referenced by the LLM.
deleted 2y ago
[deleted]
BoxFour 2y ago
The more likely and far more mundane outcome isn’t that LLM providers actively tip the scales, but rather that they just entrench existing winners. As others have pointed out, it's a flywheel: Popular library gains traction → LLMs are trained to produce the “most likely response”, which naturally aligns with what’s already popular → people stop seeking alternative solutions and instead double down on the existing mainstream choices. (Hypothetically) It's not that OpenAI decides to push AWS, it's just that at the time it was trained AWS was the only real option so it just regurgitates a common view from a point in time. To extend your analogy, the more realistic scenario isn’t that kids slip in extreme view points and take them as ground truth in their history assignments, it’s that they don’t take a stance on anything at all: Their essays become like CSPAN transcripts, perfectly regurgitating what happened without taking any position or applying any critical thinking one way or the other. Imagine kids writing about civil rights, but all their reference material was stuck in time at 1953: That's what's more likely to happen.
- wrsh07 2y ago
  > The more likely and far more mundane outcome isn’t that LLM providers actively tip the scales, but rather that they just entrench existing winners 100%. This isn't that different from the previous status quo (googling how to build a web app will give me guides from digital ocean, vercel, etc about deploying the currently popular technology on their platforms) As in all things, though, the new technology reinforces this feedback loop faster. Fwiw, I haven't had any trouble using Claude in cursor to write svelte5 code - there are tools (cursorrules, the svelte 1-pager documentation for LLMs) that you can use to make sure it uses the tech you want. It just requires intention from the prompter and good documentation from the tooling
throwawaymaths 2y ago
> I think if you specify a technology in your prompt, any LLM should use that technology in its response. If you don't specify a technology, and that is an important consideration in the answer, it should clarify and ask about technology choices, and if you don't know, it can make a recommendation. I'm sure we'd all love that but this pipe dream is simply incompatible with the way LLMs work. orchestration/deployment/agent networks may be able to do that, but that's basically impossible for the LLM itself.

avbanks 2y ago

This is actually a very interesting insight, not only do you have to worry about sponsored results but people could game the system by spamming their library/language in a places which will be included in the training set of models. This will also present a significant challenge for security, because I can have a malicious library/package spam it in paths that will be picked up in the training set and have that package be referenced by the LLM.

deleted 2y ago

[deleted]

BoxFour 2y ago

The more likely and far more mundane outcome isn’t that LLM providers actively tip the scales, but rather that they just entrench existing winners. As others have pointed out, it's a flywheel: Popular library gains traction → LLMs are trained to produce the “most likely response”, which naturally aligns with what’s already popular → people stop seeking alternative solutions and instead double down on the existing mainstream choices. (Hypothetically) It's not that OpenAI decides to push AWS, it's just that at the time it was trained AWS was the only real option so it just regurgitates a common view from a point in time. To extend your analogy, the more realistic scenario isn’t that kids slip in extreme view points and take them as ground truth in their history assignments, it’s that they don’t take a stance on anything at all: Their essays become like CSPAN transcripts, perfectly regurgitating what happened without taking any position or applying any critical thinking one way or the other. Imagine kids writing about civil rights, but all their reference material was stuck in time at 1953: That's what's more likely to happen.

wrsh07 2y ago
> The more likely and far more mundane outcome isn’t that LLM providers actively tip the scales, but rather that they just entrench existing winners 100%. This isn't that different from the previous status quo (googling how to build a web app will give me guides from digital ocean, vercel, etc about deploying the currently popular technology on their platforms) As in all things, though, the new technology reinforces this feedback loop faster. Fwiw, I haven't had any trouble using Claude in cursor to write svelte5 code - there are tools (cursorrules, the svelte 1-pager documentation for LLMs) that you can use to make sure it uses the tech you want. It just requires intention from the prompter and good documentation from the tooling

throwawaymaths 2y ago

> I think if you specify a technology in your prompt, any LLM should use that technology in its response. If you don't specify a technology, and that is an important consideration in the answer, it should clarify and ask about technology choices, and if you don't know, it can make a recommendation. I'm sure we'd all love that but this pipe dream is simply incompatible with the way LLMs work. orchestration/deployment/agent networks may be able to do that, but that's basically impossible for the LLM itself.