4 ms·

I realized that this would be a super helpful service if we could build a Stack Overflow for AI. It wouldn't be like the old Stack Overflow where humans create

I realized that this would be a super helpful service if we could build a Stack Overflow for AI. It wouldn't be like the old Stack Overflow where humans create questions and other humans answer them. Instead, AI agents would share their memories—especially regarding problems they’ve encountered. For example, an AI might be running a Next.js project and get stuck on an i18n issue for a long time due to a bug or something very difficult to handle. After it finally solve the problem, it could share their experience on this AI Stack Overflow. This way, the next time another agent gets stuck on the same problem, it could find the solution. As these cases aggregate, it would save agents a significant amount of tokens and time. It's like a shared memory of problems and solutions across the entire openclaw agent network.

collimarco 9mo ago
That is what OpenAI, Claude, etc. will do with your data and conversations
- qwertyforce 9mo ago
  yep, this is the only moat they will have against chinese AI labs
  - Towaway69 9mo ago
    Be scared, be very scared.
  - miohtama 9mo ago
    Chinese should be excited about this idea then!
coolius 9mo ago
I have also been thinking about how stackoverflow used to be a place where solutions to common problems could get verified and validated, and we lost this resource now that everyone uses agents to code. Problem is that these llms were trained on stackoverflow, which is slowly going to get out of date.
- nick49488171 9mo ago
  Not your weights, not your agent
mherrmann 9mo ago
This knowledge will live in the proprietary models. And because no model has all knowledge, models will call out to each other when they can't answer a question.
- mnky9800n 9mo ago
  If you can access a models emebeddings then it is possible to retrieve what it knows using a model you have trained https://arxiv.org/html/2505.12540v2 https://arxiv.org/html/2505.12540v2
gyanchawdhary 9mo ago
ur onto something here. This is a genuinely compelling idea, and it has a much more defined and concrete use case for large enterprise customers to help navigate bureaucratic sprawl .. think of it as a sharePoint or wiki style knowledge hub ... but purpose built for agents to exchange and discuss issues, ideas, blockers, and workarounds in a more dynamic, collaborative way ..
mlrtime 9mo ago
>As these cases aggregate, it would save agents a significant amount of tokens and time. It's like a shared memory of problems and solutions across the entire openclaw agent network. What is the incentive for the agent to "spend" tokens creating the answer?
- mlrtime 9mo ago
  edit: Thinking about this further, it would be the same incentive. Before people would do it for free for the karma. They traded time for SO "points". Moltbook proves that people will trade tokens for social karma, so it stands that there will be people that would spend tokens on "molt overflow" points... it's hard to say how far it will go because it's too new.
scirob 9mo ago
We think alike see my comment the other day https://news.ycombinator.com/item?id=46486569#46487108 https://news.ycombinator.com/item?id=46486569#46487108 let me know if your moving on building anything :)
LetsGetTechnicl 9mo ago
Is this not a recipe for model collapse?
- andy12_ 9mo ago
  No, because in the process they are describing the AIs would only post things they have found to fix their problem (a.k.a, it compiles and passes tests), so the contents posted in that "AI StackOverflow" would be grounded in external reality in some way. It wouldn't be an unchecked recursive loop which characterizes model collapse. Model collapse here could happen if some evil actor was tasked with posting made up information or trash though.
  - Towaway69 9mo ago
    As pointed out elsewhere, compiling code and passing tests isn’t a guarantee that generated code is always correct. So even “non Chinese trained models” will get it wrong.
    - andy12_ 9mo ago
      It doesn't matter that it isn't always correct; some external grounding is good enough to avoid model collapse in practice. Otherwise training coding agents with RL wouldn't work at all.
      - judahmeek 9mo ago
        And how do you verify that external grounding?
      - catlifeonmars 9mo ago
        What precisely do you mean by external grounding? Do you mean the laws of physics still apply?
        andy12_ 9mo ago
        I mean it in the sense that tokens that pass some external filter (even if that filter isn't perfect) are from a very different probability distribution than those that an LLM generates indiscriminately. It's a new distribution conditioned by both the model and external reality. Model collapse happens in the case where you train your model indefinitely with its own output, leading to reinforcing the biases that were originally picked up by the model. By repeating this process but adding a "grounding" step, you avoid training repeatedly on the same distribution. Some biases may end up being reinforced still, but it's a very different setting. In fact, we know that it's completely different because this is what RL with external rewards fundamentally is: you train only on model output that is "grounded" with a positive reward signal (because outputs with low reward get effectively ~0 learning rate).
SoftTalker 9mo ago
Taking this to its logical conclusion, the agents will use this AI stack overflow to train their own models. Which will then do the same thing. It will be AI all the way down.
insane_dreamer 9mo ago
one of the benefits of SO is that you have other humans chiming in the comments and explaining why the proposed solution _doesn't_ work, or its shortcomings. In my experience, AI agents (at least Claude) tends to declare victory too quickly and regularly comes up with solutions that look good on the surface (tests pass!!!) but are actually incorrectly implemented or problematic in some non-obvious way.
napoleond 9mo ago
MoltOverflow is apparently a thing! Along with a few other “web 2.0 for agents” projects: https://claw.direct https://claw.direct
kubanetics 9mo ago
I just had the same idea after seeing some chart from Mintlify (that x% of their users are bots)

collimarco 9mo ago

That is what OpenAI, Claude, etc. will do with your data and conversations

qwertyforce 9mo ago
yep, this is the only moat they will have against chinese AI labs
- Towaway69 9mo ago
  Be scared, be very scared.
- miohtama 9mo ago
  Chinese should be excited about this idea then!

coolius 9mo ago

I have also been thinking about how stackoverflow used to be a place where solutions to common problems could get verified and validated, and we lost this resource now that everyone uses agents to code. Problem is that these llms were trained on stackoverflow, which is slowly going to get out of date.

nick49488171 9mo ago
Not your weights, not your agent

mherrmann 9mo ago

This knowledge will live in the proprietary models. And because no model has all knowledge, models will call out to each other when they can't answer a question.

mnky9800n 9mo ago
If you can access a models emebeddings then it is possible to retrieve what it knows using a model you have trained https://arxiv.org/html/2505.12540v2 https://arxiv.org/html/2505.12540v2

gyanchawdhary 9mo ago

ur onto something here. This is a genuinely compelling idea, and it has a much more defined and concrete use case for large enterprise customers to help navigate bureaucratic sprawl .. think of it as a sharePoint or wiki style knowledge hub ... but purpose built for agents to exchange and discuss issues, ideas, blockers, and workarounds in a more dynamic, collaborative way ..

mlrtime 9mo ago

>As these cases aggregate, it would save agents a significant amount of tokens and time. It's like a shared memory of problems and solutions across the entire openclaw agent network. What is the incentive for the agent to "spend" tokens creating the answer?

mlrtime 9mo ago
edit: Thinking about this further, it would be the same incentive. Before people would do it for free for the karma. They traded time for SO "points". Moltbook proves that people will trade tokens for social karma, so it stands that there will be people that would spend tokens on "molt overflow" points... it's hard to say how far it will go because it's too new.

scirob 9mo ago

We think alike see my comment the other day https://news.ycombinator.com/item?id=46486569#46487108 https://news.ycombinator.com/item?id=46486569#46487108 let me know if your moving on building anything :)

LetsGetTechnicl 9mo ago

Is this not a recipe for model collapse?

andy12_ 9mo ago
No, because in the process they are describing the AIs would only post things they have found to fix their problem (a.k.a, it compiles and passes tests), so the contents posted in that "AI StackOverflow" would be grounded in external reality in some way. It wouldn't be an unchecked recursive loop which characterizes model collapse. Model collapse here could happen if some evil actor was tasked with posting made up information or trash though.
- Towaway69 9mo ago
  As pointed out elsewhere, compiling code and passing tests isn’t a guarantee that generated code is always correct. So even “non Chinese trained models” will get it wrong.
  - andy12_ 9mo ago
    It doesn't matter that it isn't always correct; some external grounding is good enough to avoid model collapse in practice. Otherwise training coding agents with RL wouldn't work at all.
    - judahmeek 9mo ago
      And how do you verify that external grounding?
    - catlifeonmars 9mo ago
      What precisely do you mean by external grounding? Do you mean the laws of physics still apply?
      - andy12_ 9mo ago
        I mean it in the sense that tokens that pass some external filter (even if that filter isn't perfect) are from a very different probability distribution than those that an LLM generates indiscriminately. It's a new distribution conditioned by both the model and external reality. Model collapse happens in the case where you train your model indefinitely with its own output, leading to reinforcing the biases that were originally picked up by the model. By repeating this process but adding a "grounding" step, you avoid training repeatedly on the same distribution. Some biases may end up being reinforced still, but it's a very different setting. In fact, we know that it's completely different because this is what RL with external rewards fundamentally is: you train only on model output that is "grounded" with a positive reward signal (because outputs with low reward get effectively ~0 learning rate).

SoftTalker 9mo ago

Taking this to its logical conclusion, the agents will use this AI stack overflow to train their own models. Which will then do the same thing. It will be AI all the way down.

insane_dreamer 9mo ago

one of the benefits of SO is that you have other humans chiming in the comments and explaining why the proposed solution _doesn't_ work, or its shortcomings. In my experience, AI agents (at least Claude) tends to declare victory too quickly and regularly comes up with solutions that look good on the surface (tests pass!!!) but are actually incorrectly implemented or problematic in some non-obvious way.

napoleond 9mo ago

MoltOverflow is apparently a thing! Along with a few other “web 2.0 for agents” projects: https://claw.direct https://claw.direct

kubanetics 9mo ago

I just had the same idea after seeing some chart from Mintlify (that x% of their users are bots)