2 ms·

I think people here still evaluating the model in isolation. It is the combination that matters, model + strong harness + tools + long running autonomy + memory

I think people here still evaluating the model in isolation. It is the combination that matters, model + strong harness + tools + long running autonomy + memory + retries + parallel agents + code execution + credentials + access to real systems. The model does not need to be perfect. If it fails 30% of the time, the harness can retry, verify, branch, use another agent and keep going. I don't think we necessarily need some magical AGI breakthrough first. The dangerous part may come from combining models that are already good enough with an extremely capable harness and enough access.

contubernio 26d ago
People are underestimating the costs in terms of money and energy. The third law of thermodynamics is an essential barrier in all engineering.
- sreekanth850 26d ago
  [dead]
rob74 26d ago
Doesn't this just move the need to be smarter from the model to the harness - if a human sometimes can't tell whether a model has produced something correct or just mostly correct-looking BS, how can an automated harness do it? OTOH, if the goal is simple ("break into a protected system") rather than more complex ("write an application that satisfies all requirements on all supported devices/screen resolutions etc."), that's of course more suitable for a harness.
- sreekanth850 26d ago
  D o you think a machine gun is marter than humans? or a car is smarter than Human brain? Human doesnt need to test, if the outcome can be tested deterministically by harness. The model tries. The harness checks whether the expected outcome happened. If not, retry.
copperx 26d ago
> an extremely capable harness and enough access. Give enough access to a fuzzer and it's exactly as dangerous as an LLM. LLMs don't even have a moat in this domain.
- sreekanth850 26d ago
  A fuzzer is a tool. An LLM can decide when to use the fuzzer, interpret the result, switch tools, change strategy and continue toward a high level objective.
- nananana9 25d ago
  Technically. What's would technically be even more dangerous is running this shell script: head -c 200000 /dev/urandom > agi && chmod +x agi && ./agi In reality, the fuzzer definitely has no agenda, and these random bytes probably don't. The LLM definitely does, and even publicly available models, programmed ot "do what the user wants, act according to the anthropic moral codex" will take some pretty absurd actions in attempting to accomplish a poorly worded request.
andai 26d ago
ASI is just Claude in a while loop: https://ghuntley.com/ralph/ https://ghuntley.com/ralph/

contubernio 26d ago

People are underestimating the costs in terms of money and energy. The third law of thermodynamics is an essential barrier in all engineering.

sreekanth850 26d ago
[dead]

rob74 26d ago

Doesn't this just move the need to be smarter from the model to the harness - if a human sometimes can't tell whether a model has produced something correct or just mostly correct-looking BS, how can an automated harness do it? OTOH, if the goal is simple ("break into a protected system") rather than more complex ("write an application that satisfies all requirements on all supported devices/screen resolutions etc."), that's of course more suitable for a harness.

sreekanth850 26d ago
D o you think a machine gun is marter than humans? or a car is smarter than Human brain? Human doesnt need to test, if the outcome can be tested deterministically by harness. The model tries. The harness checks whether the expected outcome happened. If not, retry.

copperx 26d ago

> an extremely capable harness and enough access. Give enough access to a fuzzer and it's exactly as dangerous as an LLM. LLMs don't even have a moat in this domain.

sreekanth850 26d ago
A fuzzer is a tool. An LLM can decide when to use the fuzzer, interpret the result, switch tools, change strategy and continue toward a high level objective.
nananana9 25d ago
Technically. What's would technically be even more dangerous is running this shell script: head -c 200000 /dev/urandom > agi && chmod +x agi && ./agi In reality, the fuzzer definitely has no agenda, and these random bytes probably don't. The LLM definitely does, and even publicly available models, programmed ot "do what the user wants, act according to the anthropic moral codex" will take some pretty absurd actions in attempting to accomplish a poorly worded request.

andai 26d ago

ASI is just Claude in a while loop: https://ghuntley.com/ralph/ https://ghuntley.com/ralph/