2 ms·

I've long suspected that LLMs will just output pure bits eventually

by cynicalpeace 3mo ago

nicebyte 3mo ago
are you implying that text is impure bits?
- cynicalpeace 3mo ago
  when the model predicts the next token, it doesn't predict the next bit. I'm saying, eventually, it will... perhaps :)
hankbond 3mo ago
well they can natively converse in base64
rytill 3mo ago
Why would this be the case when the text that produces binaries (code) is usually both more token efficient and vastly more effectively organized for modification/extension? Unless by bits you just mean text in general, or any data since it’s all bits, in which case what you’re saying is trivially already true. It seems like you’re saying that long term LLMs will output pure machine code as the most effective way to use them.
- cynicalpeace 3mo ago
  token efficiency could become irrelevant in the long term
  - rytill 3mo ago
    What about the other thing that’s the actually important part of my reply, modifiability/extensibility? You just chose the weaker of the two factors I provided. And didn’t even provide a convincing argument related to it. You also didn’t clarify your position at all. Explain why there is any benefit to outputting raw machine instructions compared to writing shorter, more interpretable, modifiable, extensible code and then using a compiler to turn it into machine code. Why are agents not going to use compilers in the future?
    - cynicalpeace 3mo ago
      I'm not convinced text is more modifiable and extensible than straight bits for a "superintelligence" The benefit would be that the AI would have true hardware access. It would be unconstrained by gatekeeper protocols. Would be very powerful, and thereby, potentially dangerous.
zerr 3mo ago
Or just replace all of the apps? Why would a user use apps if everything (eventually) can be achieved with an AI?
- cynicalpeace 3mo ago
  i think the main blocker to this are the owners of the App Stores themselves. Apple won't let this happen if it can help it.

I've long suspected that LLMs will just output pure bits eventually

nicebyte 3mo ago

are you implying that text is impure bits?

cynicalpeace 3mo ago
when the model predicts the next token, it doesn't predict the next bit. I'm saying, eventually, it will... perhaps :)

hankbond 3mo ago

well they can natively converse in base64

rytill 3mo ago

Why would this be the case when the text that produces binaries (code) is usually both more token efficient and vastly more effectively organized for modification/extension? Unless by bits you just mean text in general, or any data since it’s all bits, in which case what you’re saying is trivially already true. It seems like you’re saying that long term LLMs will output pure machine code as the most effective way to use them.

cynicalpeace 3mo ago
token efficiency could become irrelevant in the long term
- rytill 3mo ago
  What about the other thing that’s the actually important part of my reply, modifiability/extensibility? You just chose the weaker of the two factors I provided. And didn’t even provide a convincing argument related to it. You also didn’t clarify your position at all. Explain why there is any benefit to outputting raw machine instructions compared to writing shorter, more interpretable, modifiable, extensible code and then using a compiler to turn it into machine code. Why are agents not going to use compilers in the future?
  - cynicalpeace 3mo ago
    I'm not convinced text is more modifiable and extensible than straight bits for a "superintelligence" The benefit would be that the AI would have true hardware access. It would be unconstrained by gatekeeper protocols. Would be very powerful, and thereby, potentially dangerous.

zerr 3mo ago

Or just replace all of the apps? Why would a user use apps if everything (eventually) can be achieved with an AI?

cynicalpeace 3mo ago
i think the main blocker to this are the owners of the App Stores themselves. Apple won't let this happen if it can help it.