5 ms·

> An LLM won't call other nodes in the organization to check when it sees that the value is unreasonable for some out-of-context reason, like yesterday was a on

> An LLM won't call other nodes in the organization to check when it sees that the value is unreasonable for some out-of-context reason, like yesterday was a one-time-only bank holiday and so the value should be 0. Why not? LLMs are the first kind of technology that can take this kind of global view. We're not making much use of it in this way just yet, but considering "out-of-context reasons" and taking a wider perspective is pretty much the defining aspect of LLMs as general-purpose AI tools. In time, I expect them to match humans on this (at least humans that care; it's not hard to match those who don't). I do agree on the liability angle. This increasingly seems to be the main value a human brings to the table. It's not a new trend, though. See e.g. medicine, architecture, civil engineering - licensed professionals aren't doing the bulk of the work, but they're in the loop and well-compensated for verifying and signing off on the work done by less-paid technicians.

ben_w 1y ago
> I do agree on the liability angle. This increasingly seems to be the main value a human brings to the table. It's not a new trend, though. See e.g. medicine, architecture, civil engineering - licensed professionals aren't doing the bulk of the work, but they're in the loop and well-compensated for verifying and signing off on the work done by less-paid technicians. Ironic that this liability issue is one of the big ways that "software engineer" isn't like any other kind of engineer. My university was saying as much 20 years ago, well before GenAI.
- TeMPOraL 1y ago
  > Ironic that this liability issue is one of the big ways that "software engineer" isn't like any other kind of engineer. In context discussed here, it generally is. Licensed engineers are independent (or at least supposed to be), which adds an otherwise interesting cross-organizational dimension, but in terms of having human in a loop, an employee with the right set of skills, deputized for this purpose by the company, is sufficient to make the organization compliant and let the liability flow elsewhere. That can be a software engineer, for matters relevant to tech solutions, but in different areas/contexts, it doesn't have to be an engineer (licensed or otherwise) at all.
saulpw 1y ago
> considering "out-of-context reasons" and taking a wider perspective is pretty much the defining aspect of LLMs as general-purpose AI tools. "out-of-context" literally means that the reason isn't in its context. Even if it can make the leap that the number should be zero if it's a bank holiday, how would an LLM know that yesterday was a one-off bank holiday? A human would only know through their lived experience that the markets were shut down, the news was making a big deal over it, etc. It's the same problem using cheap human labor in a different region of the world for this kind of thing; they can perform the mechanical task, but they don't have the context to detect the myriad of ways it can go subtly wrong.
- ben_w 1y ago
  That kind of knowledge is present in the training set, doesn't need to be in the context or system prompt. That said, I too would only use an LLM today in the same kinds of role that five years ago would be outsourced to a different culture. Culture, not even language: this is how you get the difference between "biscuits and gravy" as understood in the UK vs in the USA.
  - TeMPOraL 1y ago
    LLMs handle major trappings of culture just fine. As long as a culture has enough of a footprint in terms of written words, the LLM probably knows it better than any single individual, even though it has not lived it.
    - alganet 1y ago
      No, it does not know culture. And no, it can't handle talking about it. Ask an LLM "Can you compare egyptian mythology with aliens?" and they will happily do it: https://imgur.com/a/jfikuEO https://imgur.com/a/jfikuEO That's an offensive, pseudoscientific view on egyptian culture shunned by academics. Even ChatGPT "Critical Viewpoint" section (a small part of a large bullshit response) _still_ entertains offensive ideas: https://imgur.com/a/RNiMmJZ https://imgur.com/a/RNiMmJZ They should have answered that such comparisons are potentially offensive, and explained why academia thinks so, _before_ spilling out nonsense.

ben_w 1y ago

> I do agree on the liability angle. This increasingly seems to be the main value a human brings to the table. It's not a new trend, though. See e.g. medicine, architecture, civil engineering - licensed professionals aren't doing the bulk of the work, but they're in the loop and well-compensated for verifying and signing off on the work done by less-paid technicians. Ironic that this liability issue is one of the big ways that "software engineer" isn't like any other kind of engineer. My university was saying as much 20 years ago, well before GenAI.

TeMPOraL 1y ago
> Ironic that this liability issue is one of the big ways that "software engineer" isn't like any other kind of engineer. In context discussed here, it generally is. Licensed engineers are independent (or at least supposed to be), which adds an otherwise interesting cross-organizational dimension, but in terms of having human in a loop, an employee with the right set of skills, deputized for this purpose by the company, is sufficient to make the organization compliant and let the liability flow elsewhere. That can be a software engineer, for matters relevant to tech solutions, but in different areas/contexts, it doesn't have to be an engineer (licensed or otherwise) at all.

saulpw 1y ago

> considering "out-of-context reasons" and taking a wider perspective is pretty much the defining aspect of LLMs as general-purpose AI tools. "out-of-context" literally means that the reason isn't in its context. Even if it can make the leap that the number should be zero if it's a bank holiday, how would an LLM know that yesterday was a one-off bank holiday? A human would only know through their lived experience that the markets were shut down, the news was making a big deal over it, etc. It's the same problem using cheap human labor in a different region of the world for this kind of thing; they can perform the mechanical task, but they don't have the context to detect the myriad of ways it can go subtly wrong.

ben_w 1y ago
That kind of knowledge is present in the training set, doesn't need to be in the context or system prompt. That said, I too would only use an LLM today in the same kinds of role that five years ago would be outsourced to a different culture. Culture, not even language: this is how you get the difference between "biscuits and gravy" as understood in the UK vs in the USA.
- TeMPOraL 1y ago
  LLMs handle major trappings of culture just fine. As long as a culture has enough of a footprint in terms of written words, the LLM probably knows it better than any single individual, even though it has not lived it.
  - alganet 1y ago
    No, it does not know culture. And no, it can't handle talking about it. Ask an LLM "Can you compare egyptian mythology with aliens?" and they will happily do it: https://imgur.com/a/jfikuEO https://imgur.com/a/jfikuEO That's an offensive, pseudoscientific view on egyptian culture shunned by academics. Even ChatGPT "Critical Viewpoint" section (a small part of a large bullshit response) _still_ entertains offensive ideas: https://imgur.com/a/RNiMmJZ https://imgur.com/a/RNiMmJZ They should have answered that such comparisons are potentially offensive, and explained why academia thinks so, _before_ spilling out nonsense.