5 ms·

You’d think ai models would be good enough to write code in pure binary. Python is for humans. Take the human out there’s no point in it.

by asdff 2y ago

gazchop 2y ago
You think something which gets 70% of stuff right on a very good day when the planets are all aligned can write assembly that works? I have news for you.
- asdff 2y ago
  I think it sure can. Randomly modify a binary function over a million potential permutations and measure the distribution of output, subset from the distribution the binary function(s) that output ahead of your threshold. Should give you a fast function that does what you want with absolutely indescribable code, which isn’t an issue if it does what you want with reasonable performance.
koe123 2y ago
I suppose the issue is a data problem, there being relatively little high quality data explaining how things should be solved in binary. As such making the learning mapping between prompt (english) and good solution (binary) difficult.
- ninetyninenine 2y ago
  The data is easily generated by compiling the code.
  - 0xCMP 2y ago
    But compiled code loses a lot of the "extra" data. Also these are "language" models so I would be surprised if training on binaries was much more efficient versus writing in some kind of language. Besides, how do you even check the result now without running untrusted code? Every run of the model you need to reverse-engineer the binary?
- asdff 2y ago
  You don’t even need a training set imo just enough permutations to get a distribution of putative functions and outputs.
raincole 2y ago
It's possible that one day they'll be. But to me it's very obvious why LLMs are better at writing human-readable code than binary code. The hype around LLMs come from the fact that you can command them in natural language. To make it possible, LLMs have to be trained mostly on natural language corpus. And human-readable code is closer to natural language than binary code.
probably_wrong 2y ago
With all due respect, I think that's a terrible idea for a couple reasons. First, a model that produces binary code would be impossible to vet. This is already an issue with "trusting trust" in compilers [1], and would be made worse in the case of companies that constantly tweak their black-box models however they like. At least my compiler can work offline. But then there's the idea of programming as "turn data into data" that forgets that software has to interface with humans too. If I ask my LLM to give me sentencing guidelines that are "fair" given a set of parameters, no one on Earth should trust a binary they cannot check themselves. In fact, those affected may even have a constitutional right to source code. [1] https://dl.acm.org/doi/10.1145/358198.358210 https://dl.acm.org/doi/10.1145/358198.358210
- asdff 2y ago
  Imo you wouldn’t even have to check the binary. Only that the output is what you expect and the performance is reasonable. No different than say agriculture where we measure success of a new varietal on output and resource use and have a very dim understanding of the inner workings that might have led to those observables. Or to take it further, you hire a consultant to produce some work in a given amount of time. You accept the work over time, you don’t feel the need to ask the consultant to turn over all their notes or to plug their brain into a mind reader machine to vet the source. They could indeed be lying and take whatever they learned about your company elsewhere, but we have no protection over that possibility lacking a mind reading instrument today. And society also doesn’t collapse today given that risk.

You’d think ai models would be good enough to write code in pure binary. Python is for humans. Take the human out there’s no point in it.

by asdff 2y ago

You’d think ai models would be good enough to write code in pure binary. Python is for humans. Take the human out there’s no point in it.

gazchop 2y ago

You think something which gets 70% of stuff right on a very good day when the planets are all aligned can write assembly that works? I have news for you.

asdff 2y ago
I think it sure can. Randomly modify a binary function over a million potential permutations and measure the distribution of output, subset from the distribution the binary function(s) that output ahead of your threshold. Should give you a fast function that does what you want with absolutely indescribable code, which isn’t an issue if it does what you want with reasonable performance.

koe123 2y ago

I suppose the issue is a data problem, there being relatively little high quality data explaining how things should be solved in binary. As such making the learning mapping between prompt (english) and good solution (binary) difficult.

ninetyninenine 2y ago
The data is easily generated by compiling the code.
- 0xCMP 2y ago
  But compiled code loses a lot of the "extra" data. Also these are "language" models so I would be surprised if training on binaries was much more efficient versus writing in some kind of language. Besides, how do you even check the result now without running untrusted code? Every run of the model you need to reverse-engineer the binary?
asdff 2y ago
You don’t even need a training set imo just enough permutations to get a distribution of putative functions and outputs.

raincole 2y ago

It's possible that one day they'll be. But to me it's very obvious why LLMs are better at writing human-readable code than binary code. The hype around LLMs come from the fact that you can command them in natural language. To make it possible, LLMs have to be trained mostly on natural language corpus. And human-readable code is closer to natural language than binary code.

probably_wrong 2y ago

With all due respect, I think that's a terrible idea for a couple reasons. First, a model that produces binary code would be impossible to vet. This is already an issue with "trusting trust" in compilers [1], and would be made worse in the case of companies that constantly tweak their black-box models however they like. At least my compiler can work offline. But then there's the idea of programming as "turn data into data" that forgets that software has to interface with humans too. If I ask my LLM to give me sentencing guidelines that are "fair" given a set of parameters, no one on Earth should trust a binary they cannot check themselves. In fact, those affected may even have a constitutional right to source code. [1] https://dl.acm.org/doi/10.1145/358198.358210 https://dl.acm.org/doi/10.1145/358198.358210

asdff 2y ago
Imo you wouldn’t even have to check the binary. Only that the output is what you expect and the performance is reasonable. No different than say agriculture where we measure success of a new varietal on output and resource use and have a very dim understanding of the inner workings that might have led to those observables. Or to take it further, you hire a consultant to produce some work in a given amount of time. You accept the work over time, you don’t feel the need to ask the consultant to turn over all their notes or to plug their brain into a mind reader machine to vet the source. They could indeed be lying and take whatever they learned about your company elsewhere, but we have no protection over that possibility lacking a mind reading instrument today. And society also doesn’t collapse today given that risk.