3 ms·

With all due respect, I think that's a terrible idea for a couple reasons. First, a model that produces binary code would be impossible to vet. This is already

With all due respect, I think that's a terrible idea for a couple reasons. First, a model that produces binary code would be impossible to vet. This is already an issue with "trusting trust" in compilers [1], and would be made worse in the case of companies that constantly tweak their black-box models however they like. At least my compiler can work offline. But then there's the idea of programming as "turn data into data" that forgets that software has to interface with humans too. If I ask my LLM to give me sentencing guidelines that are "fair" given a set of parameters, no one on Earth should trust a binary they cannot check themselves. In fact, those affected may even have a constitutional right to source code. [1] https://dl.acm.org/doi/10.1145/358198.358210 https://dl.acm.org/doi/10.1145/358198.358210

asdff 2y ago
Imo you wouldn’t even have to check the binary. Only that the output is what you expect and the performance is reasonable. No different than say agriculture where we measure success of a new varietal on output and resource use and have a very dim understanding of the inner workings that might have led to those observables. Or to take it further, you hire a consultant to produce some work in a given amount of time. You accept the work over time, you don’t feel the need to ask the consultant to turn over all their notes or to plug their brain into a mind reader machine to vet the source. They could indeed be lying and take whatever they learned about your company elsewhere, but we have no protection over that possibility lacking a mind reading instrument today. And society also doesn’t collapse today given that risk.