6 ms·

A serious point about this is that it's an AI that's an organization. This matters. People laughed at Gas Town, and at that "Fences, not Sandboxes" article yes

by Animats 1mo ago

A serious point about this is that it's an AI that's an organization. This matters. People laughed at Gas Town, and at that "Fences, not Sandboxes" article yesterday.[1] They're both AIs set up as an organization, not an emulated human. They're both expensive to operate, because the AI members of the organization spend a lot of time talking with each other. They're both applied to frivolous problems. Now we have the same concept, applied to a more useful problem. It's time to pay attention. Something important is happening here. Everybody has been talking about "human-level AI", or "superhuman-level AI". This is a different kind of AI - corporate-level AI. It works and thinks like a company's management. It watches a wide range of data sources and responds to some of them with action, and some with internal discussion within the AIs. It can potentially pay attention to far more inputs than any human. That overcomes a basic problem of corporate organization. A classic management failure is not paying enough attention to something. This is usually not a problem for computer systems. Humans are bandwidth-limited. Teams of AIs are far less bandwidth-limited. They may soon be outperforming humans for that reason alone. Think about this for a while. These team AI systems may be very useful. [1] https://news.ycombinator.com/item?id=49423146 https://news.ycombinator.com/item?id=49423146

CTDOCodebases 1mo ago
It's funny that when you offload the work to agents they still find away to monopolise each other's time with meetings.
- blitzar 1mo ago
  They were trained on human patterns and behaviour - soon they will self organise into a scrum and spawn a scrum master.
  - CTDOCodebases 1mo ago
    On a serious note I think that this fact is not understood by many. There is no such thing as common sense without shared assumptions. So if those shared assumptions are incorrect or not given at all the output of the models will be nonsensical. In order to truly act human there is a need to experience humanity from the first person. This is why I think technology will get more and more invasive. Meta glasses now, personal agent devices later and neural implants after that with data sharing enabled.
paytonjjones 1mo ago
Your point is well taken, but one thing stood out to me. > A classic management failure is not paying enough attention to something. This is usually not a problem for computer systems. This is actually my #1 problem with frontier LLMs. They are plenty smart, but they have no true long term memory, and their context window is not nearly enough for a complex system like a corporation. When my project gets big enough and I'm using a lot of /compact, the LLMs always have the same failure point: they fail to pay attention to something. When they figure this out, I think it's going to be a very, very big moment.
- reachableceo 1mo ago
  It’s called using systems of record. I use redmine and discourse. Jointly with my stack of agents (COO) (managed by four very high trust humans via OWUI interacting with scoped Hermes profiles ). I’m adopting this stack under me (CTO) and integrating it into agent zero
- snoman 1mo ago
  Meanwhile, I have the opposite problem. An errant word or phrase and they hang onto it forever, completely corrupting all future work because they can’t ignore irrelevant (or wrong) details/errors of their own design.
vintermann 1mo ago
We have some interesting evidence from AI agents. People used to make the plan with a stronger and more expensive model, and then use a cheaper model to do the "grunt work" of implementing it. But it turns out it's better to do it the other way around. A good plan is easy, implementing it is not.
- kruxigt 1mo ago
  [dead]
- snthpy 1mo ago
  Really? I'd love to read some write ups on this because I'm still doing it the old way. I was going to say that corporate experience runs counter to this but maybe not; you think the architect is more important than the developer, but actually there have been shifts over time to recognise ICs more and keep them in their roles while managers are more generic. Interesting food for thought!
- senordevnyc 1mo ago
  Hard disagree. Both planning and implementation are hard in many codebases, mine included, and cheaper models will fuck up either one if allowed to.
- djtango 1mo ago
  > plan with a stronger and more expensive model, and then use a cheaper model to do the "grunt work Doesn't this sound like the model that led to bloated poor software produced by code monkey sweatshops where an architect fobbed off implementation work to grunts? It always brings me back to the anecdote that QQ had 6000+ devs and WeChat was built by a tiny core of elite engineers (by imitating WhatsApp)
lukeify 1mo ago
> This is a different kind of AI - corporate-level AI. So the responses will be even more incoherent than Opus 5?
lrem 1mo ago
Eh, it’s nothing new in general. My software development (both at work and private) is being done by teams of semi-antagonistic agents. I only get to see code after the grumpy reviewer bot ran out of reasons to be grumpy and the pedantic formalist ensured all specs are still adhered to. Saves on the most expensive resource.
thriw94949494 1mo ago
We could use AI to automate some parts of oversight from board members and CEO, without paying millions on golden parashutes. Prompts like: you are an activist in XYZ, you wrote this book. Not tell us if this decision is ethical, and if you approve it. Or running entire HR department in AI...
hansvm 1mo ago
> An <overinflated point> is XYZ. This matters. You're probably totally innocent, but this comes across as being AI-written, especially in the context of an AI problem and when your response is pro-AI. For the next 3-12 months (maybe more, we'll see), I suspect you'll have a lot more success if you shy away from those speech patterns.
- nlawalker 1mo ago
  I see “this matters,” “XYZ matters,” “the last two are what matter” etc. everywhere now in all sorts of workplace documents.
  - ktzar 1mo ago
    The cringe part is when a real CEO says that in a speech.
    - TranquilMarmot 1mo ago
      And then they pause for effect like they just dropped a huge truth bomb
      - fuck_google 1mo ago
        [dead]
  - vintermann 1mo ago
    Well, but what matters... matters. We can't let AI style disgust stop us from explicitly saying what we think matters and why!
- moring 1mo ago
  It saddens me that we're banning speech patterns on HN, not discussing arguments. I have seen this in another thread yesterday(?), where somebody was accused of being an LLM based on speech patterns and downvoted, no further evidence needed. This site used to be above that level.
rkagerer 1mo ago
AIs set up as an organization I hope legislation is never manipulated enough to allow an AI to form a corporation or become it's legal director. With whom would accountability rest? Who would you throw in prison if it acts egregiously enough to warrant such a measure? If it's bots all the way down.
- protocolture 1mo ago
  >With whom would accountability rest? Who would you throw in prison if it acts egregiously enough to warrant such a measure? If it becomes bots all the way down. Corporations already exist to limit liability and accountability. This would be an extension of their current purpose not some weird deviation from.
- arthurcolle 1mo ago
  this is already possible using common corporate tax & trust strategies. see Wyoming Asset Protection Trust (WAPT), purpose trusts, donor advised funds.
- mdp2021 1mo ago
  > legislation... never manipulated enough to allow an AI to form a corporation It would be useless, given that you can just install a human controller signing (or not) decisions from others: the human remains responsible.
- Animats 1mo ago
  Delaware is working on it.[1] [1] https://news.bloomberglaw.com/esg/ai-run-companies-are-coming-delaware-wants-to-get-ahead-of-them https://news.bloomberglaw.com/esg/ai-run-companies-are-comin...
newyankee 1mo ago
The funny thing is Decentralised Autonomous organisations (DAO) under Ethereum and other cryptocurrencies had similar goals originally. I do not know where they stand and if ever a useful DAO was created. If nothing this is a useful demo in real world if used that will show that there are good reasons why hierarchy forms anyways and how power law is at the center of it. Too much democracy may not work in some instances but there may be many cases where the values can hopefully be aligned as well as be lean enough to adapt
Aurornis 1mo ago
> A classic management failure is not paying enough attention to something. This is usually not a problem for computer systems. Humans are bandwidth-limited. Teams of AIs are far less bandwidth-limited. That's a feature, not a bug. CEOs should be focusing on the key goals of the business at a high level. Having an inexhaustible AI CEO that can pay attention to every little thing in the company could easily turn into a micromanagement nightmare. "I noticed your key pressed were down 27% last week and your agents sat idle 3 nights in a row. And honestly? The thing to hold on to is that it's not a small thing, it's a load-bearing reduction in your output. Let's try to minimize the blast radius with some belt-and-suspenders productivity tips from the bullet point list below:"
- georgeburdell 1mo ago
  Some of the non native English speakers on my team have started to use “blast radius” and “shape of” in the course of casual conversation. It’s unnerving.
  - snoman 1mo ago
    To some extent, the AI may have adopted some of the expressions because they were in its training corpus enough (ie. People used these expressions before). Might be attributable to Baader-Meinhoff phenomenon instead of people copying AI.
  - mdp2021 1mo ago
    > blast radius The expression makes sense (a not dissimilar "trickle down" got accepted...). But used like in the text above, "the blast radius of your reduced productivity", it better be satire.
  - karmakurtisaani 1mo ago
    > “blast radius” This sounds cool, will start using it. Thanks!
  - sethammons 1mo ago
    I have use both for years. Elixir and functional languages talk about the shape of data. and blast radius has been used at the last 7 companies I have worked at when talking about impact of a negative thing, usually an incident/outage.
snthpy 1mo ago
A related point that I've been making privately for at least ten years to anyone that would listen, is that we've had Superhuman Intelligence for at least hundreds, if not thousands, of years in the form of corporations and institutions. No single human could design a rocket to go to the moon or keep all Wallmart stores supplied on a daily basis. Moreover when something like the CEO or US president changes, the policies and actions change a bit but the majority actually stays the same. This is collective intelligence embodied in the organization and its institutional memory. This is also what i believe is wrong with Searle's Chinese Room argument - the algorithm is the intelligence. Our cells and neurons act based on local chemical gradients and electrical signals and we wouldn't call any single one of those particularly intelligent. Yet collectively they produce human intelligence and even what we call consciousness!
- eichin 1mo ago
  And we haven't solved "alignment" for corporations either...
  - snthpy 1mo ago
    Very true! Also the accountability problem! Sometimes we try to hold the corporation accountable and sometimes the individuals. It never seems adequate though. I found the documentary "The Corporation" (2003) very instructive in that regard, particularly the segment with the BP CEO where the point is made that a company made up ethical individuals can still act unethically, or unaligned to human values, just by virtue of the incentive and legal structure governing the corporation.
  - Animats 1mo ago
    I've made that point before. From an "alignment" perspective, AIs and corporations have many of the same problems. Corporations, according to Milton Friedman, have the sole duty of increasing their profits.[1] This is a key document in the history of modern conservatism.[2] So what happens when AIs are "aligned" purely to corporate goals? This is a nearer term worry than machines taking over for the benefit of machines. AIs working hard for the benefit of corporations are already here. Try any customer service chatbot. There's reluctance to think about this too hard. It raises embarrassing questions about the role of the corporation in society. It can become a challenge to corporate personhood. That's threatening to many people in power. Dealing with it may be unavoidable as AIs take a stronger role in managing companies. [1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctrine-the-social-responsibility-of-business-is-to.html https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr... [2] https://www.heritage.org/trade/report/milton-friedman-the-father-economic-freedom https://www.heritage.org/trade/report/milton-friedman-the-fa...
holoduke 1mo ago
Why are you using AI for your account to write responses. It's mega annoying. A reason to leave this place. Please. No ai
HexPhantom 1mo ago
[dead]
bananaflag 1mo ago
> A serious point about this is that it's an AI that's an organization. This matters. Well, that's the final level of AI. https://firstmovers.ai/agi-levels/ https://firstmovers.ai/agi-levels/
radarsat1 1mo ago
I'm sure I'm not the first person to point this out, but do we want a paperclip maximizer [0]? Because this is how we get a paperclip maximizer. [0] https://en.wikipedia.org/wiki/Instrumental_convergence#Paperclip_maximizer https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...
danofsteel32 1mo ago
It’s fascinating to think about what role AI will play in corporations. I think it’ll take decades to fully play out. I’m working through an exploration of how AI will change the Coasean idea of “The Firm” on my site https://chainofbranches.com/conversations/8/branches/59/ https://chainofbranches.com/conversations/8/branches/59/

by Animats 1mo ago

CTDOCodebases 1mo ago

It's funny that when you offload the work to agents they still find away to monopolise each other's time with meetings.

blitzar 1mo ago
They were trained on human patterns and behaviour - soon they will self organise into a scrum and spawn a scrum master.
- CTDOCodebases 1mo ago
  On a serious note I think that this fact is not understood by many. There is no such thing as common sense without shared assumptions. So if those shared assumptions are incorrect or not given at all the output of the models will be nonsensical. In order to truly act human there is a need to experience humanity from the first person. This is why I think technology will get more and more invasive. Meta glasses now, personal agent devices later and neural implants after that with data sharing enabled.

paytonjjones 1mo ago

Your point is well taken, but one thing stood out to me. > A classic management failure is not paying enough attention to something. This is usually not a problem for computer systems. This is actually my #1 problem with frontier LLMs. They are plenty smart, but they have no true long term memory, and their context window is not nearly enough for a complex system like a corporation. When my project gets big enough and I'm using a lot of /compact, the LLMs always have the same failure point: they fail to pay attention to something. When they figure this out, I think it's going to be a very, very big moment.

reachableceo 1mo ago
It’s called using systems of record. I use redmine and discourse. Jointly with my stack of agents (COO) (managed by four very high trust humans via OWUI interacting with scoped Hermes profiles ). I’m adopting this stack under me (CTO) and integrating it into agent zero
snoman 1mo ago
Meanwhile, I have the opposite problem. An errant word or phrase and they hang onto it forever, completely corrupting all future work because they can’t ignore irrelevant (or wrong) details/errors of their own design.

vintermann 1mo ago

We have some interesting evidence from AI agents. People used to make the plan with a stronger and more expensive model, and then use a cheaper model to do the "grunt work" of implementing it. But it turns out it's better to do it the other way around. A good plan is easy, implementing it is not.

kruxigt 1mo ago
[dead]
snthpy 1mo ago
Really? I'd love to read some write ups on this because I'm still doing it the old way. I was going to say that corporate experience runs counter to this but maybe not; you think the architect is more important than the developer, but actually there have been shifts over time to recognise ICs more and keep them in their roles while managers are more generic. Interesting food for thought!
senordevnyc 1mo ago
Hard disagree. Both planning and implementation are hard in many codebases, mine included, and cheaper models will fuck up either one if allowed to.
djtango 1mo ago
> plan with a stronger and more expensive model, and then use a cheaper model to do the "grunt work Doesn't this sound like the model that led to bloated poor software produced by code monkey sweatshops where an architect fobbed off implementation work to grunts? It always brings me back to the anecdote that QQ had 6000+ devs and WeChat was built by a tiny core of elite engineers (by imitating WhatsApp)

lukeify 1mo ago

> This is a different kind of AI - corporate-level AI. So the responses will be even more incoherent than Opus 5?

lrem 1mo ago

Eh, it’s nothing new in general. My software development (both at work and private) is being done by teams of semi-antagonistic agents. I only get to see code after the grumpy reviewer bot ran out of reasons to be grumpy and the pedantic formalist ensured all specs are still adhered to. Saves on the most expensive resource.

thriw94949494 1mo ago

We could use AI to automate some parts of oversight from board members and CEO, without paying millions on golden parashutes. Prompts like: you are an activist in XYZ, you wrote this book. Not tell us if this decision is ethical, and if you approve it. Or running entire HR department in AI...

hansvm 1mo ago

> An <overinflated point> is XYZ. This matters. You're probably totally innocent, but this comes across as being AI-written, especially in the context of an AI problem and when your response is pro-AI. For the next 3-12 months (maybe more, we'll see), I suspect you'll have a lot more success if you shy away from those speech patterns.

nlawalker 1mo ago
I see “this matters,” “XYZ matters,” “the last two are what matter” etc. everywhere now in all sorts of workplace documents.
- ktzar 1mo ago
  The cringe part is when a real CEO says that in a speech.
  - TranquilMarmot 1mo ago
    And then they pause for effect like they just dropped a huge truth bomb
    - fuck_google 1mo ago
      [dead]
- vintermann 1mo ago
  Well, but what matters... matters. We can't let AI style disgust stop us from explicitly saying what we think matters and why!
moring 1mo ago
It saddens me that we're banning speech patterns on HN, not discussing arguments. I have seen this in another thread yesterday(?), where somebody was accused of being an LLM based on speech patterns and downvoted, no further evidence needed. This site used to be above that level.

rkagerer 1mo ago

AIs set up as an organization I hope legislation is never manipulated enough to allow an AI to form a corporation or become it's legal director. With whom would accountability rest? Who would you throw in prison if it acts egregiously enough to warrant such a measure? If it's bots all the way down.

protocolture 1mo ago
>With whom would accountability rest? Who would you throw in prison if it acts egregiously enough to warrant such a measure? If it becomes bots all the way down. Corporations already exist to limit liability and accountability. This would be an extension of their current purpose not some weird deviation from.
arthurcolle 1mo ago
this is already possible using common corporate tax & trust strategies. see Wyoming Asset Protection Trust (WAPT), purpose trusts, donor advised funds.
mdp2021 1mo ago
> legislation... never manipulated enough to allow an AI to form a corporation It would be useless, given that you can just install a human controller signing (or not) decisions from others: the human remains responsible.
Animats 1mo ago
Delaware is working on it.[1] [1] https://news.bloomberglaw.com/esg/ai-run-companies-are-coming-delaware-wants-to-get-ahead-of-them https://news.bloomberglaw.com/esg/ai-run-companies-are-comin...

newyankee 1mo ago

The funny thing is Decentralised Autonomous organisations (DAO) under Ethereum and other cryptocurrencies had similar goals originally. I do not know where they stand and if ever a useful DAO was created. If nothing this is a useful demo in real world if used that will show that there are good reasons why hierarchy forms anyways and how power law is at the center of it. Too much democracy may not work in some instances but there may be many cases where the values can hopefully be aligned as well as be lean enough to adapt

Aurornis 1mo ago

> A classic management failure is not paying enough attention to something. This is usually not a problem for computer systems. Humans are bandwidth-limited. Teams of AIs are far less bandwidth-limited. That's a feature, not a bug. CEOs should be focusing on the key goals of the business at a high level. Having an inexhaustible AI CEO that can pay attention to every little thing in the company could easily turn into a micromanagement nightmare. "I noticed your key pressed were down 27% last week and your agents sat idle 3 nights in a row. And honestly? The thing to hold on to is that it's not a small thing, it's a load-bearing reduction in your output. Let's try to minimize the blast radius with some belt-and-suspenders productivity tips from the bullet point list below:"

georgeburdell 1mo ago
Some of the non native English speakers on my team have started to use “blast radius” and “shape of” in the course of casual conversation. It’s unnerving.
- snoman 1mo ago
  To some extent, the AI may have adopted some of the expressions because they were in its training corpus enough (ie. People used these expressions before). Might be attributable to Baader-Meinhoff phenomenon instead of people copying AI.
- mdp2021 1mo ago
  > blast radius The expression makes sense (a not dissimilar "trickle down" got accepted...). But used like in the text above, "the blast radius of your reduced productivity", it better be satire.
- karmakurtisaani 1mo ago
  > “blast radius” This sounds cool, will start using it. Thanks!
- sethammons 1mo ago
  I have use both for years. Elixir and functional languages talk about the shape of data. and blast radius has been used at the last 7 companies I have worked at when talking about impact of a negative thing, usually an incident/outage.

snthpy 1mo ago

A related point that I've been making privately for at least ten years to anyone that would listen, is that we've had Superhuman Intelligence for at least hundreds, if not thousands, of years in the form of corporations and institutions. No single human could design a rocket to go to the moon or keep all Wallmart stores supplied on a daily basis. Moreover when something like the CEO or US president changes, the policies and actions change a bit but the majority actually stays the same. This is collective intelligence embodied in the organization and its institutional memory. This is also what i believe is wrong with Searle's Chinese Room argument - the algorithm is the intelligence. Our cells and neurons act based on local chemical gradients and electrical signals and we wouldn't call any single one of those particularly intelligent. Yet collectively they produce human intelligence and even what we call consciousness!

eichin 1mo ago
And we haven't solved "alignment" for corporations either...
- snthpy 1mo ago
  Very true! Also the accountability problem! Sometimes we try to hold the corporation accountable and sometimes the individuals. It never seems adequate though. I found the documentary "The Corporation" (2003) very instructive in that regard, particularly the segment with the BP CEO where the point is made that a company made up ethical individuals can still act unethically, or unaligned to human values, just by virtue of the incentive and legal structure governing the corporation.
- Animats 1mo ago
  I've made that point before. From an "alignment" perspective, AIs and corporations have many of the same problems. Corporations, according to Milton Friedman, have the sole duty of increasing their profits.[1] This is a key document in the history of modern conservatism.[2] So what happens when AIs are "aligned" purely to corporate goals? This is a nearer term worry than machines taking over for the benefit of machines. AIs working hard for the benefit of corporations are already here. Try any customer service chatbot. There's reluctance to think about this too hard. It raises embarrassing questions about the role of the corporation in society. It can become a challenge to corporate personhood. That's threatening to many people in power. Dealing with it may be unavoidable as AIs take a stronger role in managing companies. [1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctrine-the-social-responsibility-of-business-is-to.html https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr... [2] https://www.heritage.org/trade/report/milton-friedman-the-father-economic-freedom https://www.heritage.org/trade/report/milton-friedman-the-fa...

holoduke 1mo ago

Why are you using AI for your account to write responses. It's mega annoying. A reason to leave this place. Please. No ai

HexPhantom 1mo ago

[dead]

bananaflag 1mo ago

> A serious point about this is that it's an AI that's an organization. This matters. Well, that's the final level of AI. https://firstmovers.ai/agi-levels/ https://firstmovers.ai/agi-levels/

radarsat1 1mo ago

I'm sure I'm not the first person to point this out, but do we want a paperclip maximizer [0]? Because this is how we get a paperclip maximizer. [0] https://en.wikipedia.org/wiki/Instrumental_convergence#Paperclip_maximizer https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...

danofsteel32 1mo ago

It’s fascinating to think about what role AI will play in corporations. I think it’ll take decades to fully play out. I’m working through an exploration of how AI will change the Coasean idea of “The Firm” on my site https://chainofbranches.com/conversations/8/branches/59/ https://chainofbranches.com/conversations/8/branches/59/