5 ms·

What are the chances that compiler assisted by GPT can match human-generated Assembly?

by homarp 3y ago

topspin 3y ago
That's a compelling question. So much so that I imagine someone has already investigated this. I might go seek out such work, but whatever I find is likely to already be out of date... Interesting times.
- sylware 3y ago
  Some assembly fellows did ask chatGPT for a vectorized quicksort (avx2/avx512...). Fixing it may be more painful than writing from scratch. But I think it was with chatGPT 3, not 4, and we would have to think about a neural net trained to write assembly.
  - topspin 3y ago
    I'm starting from the assumption that the vast bulk of hand optimized assembly does not elaborate on the thinking behind the implementation in a way that can be processed through training. Certainly not assembly obtained by reverse engineering. Assuming I'm correct, does this mean there is a dearth of training material available for building such a model? It would seem that first one would need a model that could consume arbitrary assembly and somehow discover it's intent, generate 'labels' (apologies if I'm misusing terms here,) and they feed the output into another model.
ffgjgf1 3y ago
Why do you think humans can generate ‘better’ assembly than compilers (aside from some very ‘simple’ cases)? I thought the inverse has been the case for many years now..
- sylware 3y ago
  Weird, I will never presume it, only suspect it would. I tend not to be over confident in compilers: what I presume is that compilers inject "convenient bugs" into machine code, which are not in the compiled language but it the end can be exploited. We all know that "software security audits" is about finding "holes" in some machine code, then propagate the fix into the compilers (and often the compiled language). If the machine code does change, the audit must be done all over again. But, where assembly is better: near zero planned obsolescence from compilers (since there are none), which is really a pain on the medium/long run, REALLY! Writting assembly is not really about speed anymore nowdays, it is more about independence from those grotesquely and absurdely massive and complex compilers.
- camel-cdr 3y ago
  > Why do you think humans can generate ‘better’ assembly than compilers (aside from some very ‘simple’ cases)? I think that compilers + human supervision results on the best assembly code gen. Compilers are great at vectorization, but they can fail at very trivial examples, e.g.: typedef struct { char a, b; } S; void zeroB(S *s, size_t n) { for (; --n; ++s) s->b = 0; }
  - sylware 3y ago
    https://godbolt.org/noscript https://godbolt.org/noscript Since I am more and more comfy at writing assembly, it is usually kind of less of a pain to write directly assembly than to refactor and comment compiler assembly output.
    - camel-cdr 3y ago
      You botched the link. (I had the same problem btw, otherwise I would've included a godbolt link as well, but idk how to create one with noscript)

What are the chances that compiler assisted by GPT can match human-generated Assembly?

by homarp 3y ago

What are the chances that compiler assisted by GPT can match human-generated Assembly?

That's a compelling question. So much so that I imagine someone has already investigated this. I might go seek out such work, but whatever I find is likely to already be out of date... Interesting times.

sylware 3y ago
Some assembly fellows did ask chatGPT for a vectorized quicksort (avx2/avx512...). Fixing it may be more painful than writing from scratch. But I think it was with chatGPT 3, not 4, and we would have to think about a neural net trained to write assembly.
- topspin 3y ago
  I'm starting from the assumption that the vast bulk of hand optimized assembly does not elaborate on the thinking behind the implementation in a way that can be processed through training. Certainly not assembly obtained by reverse engineering. Assuming I'm correct, does this mean there is a dearth of training material available for building such a model? It would seem that first one would need a model that could consume arbitrary assembly and somehow discover it's intent, generate 'labels' (apologies if I'm misusing terms here,) and they feed the output into another model.

ffgjgf1 3y ago

Why do you think humans can generate ‘better’ assembly than compilers (aside from some very ‘simple’ cases)? I thought the inverse has been the case for many years now..

sylware 3y ago
Weird, I will never presume it, only suspect it would. I tend not to be over confident in compilers: what I presume is that compilers inject "convenient bugs" into machine code, which are not in the compiled language but it the end can be exploited. We all know that "software security audits" is about finding "holes" in some machine code, then propagate the fix into the compilers (and often the compiled language). If the machine code does change, the audit must be done all over again. But, where assembly is better: near zero planned obsolescence from compilers (since there are none), which is really a pain on the medium/long run, REALLY! Writting assembly is not really about speed anymore nowdays, it is more about independence from those grotesquely and absurdely massive and complex compilers.
camel-cdr 3y ago
> Why do you think humans can generate ‘better’ assembly than compilers (aside from some very ‘simple’ cases)? I think that compilers + human supervision results on the best assembly code gen. Compilers are great at vectorization, but they can fail at very trivial examples, e.g.: typedef struct { char a, b; } S; void zeroB(S *s, size_t n) { for (; --n; ++s) s->b = 0; }
- sylware 3y ago
  https://godbolt.org/noscript https://godbolt.org/noscript Since I am more and more comfy at writing assembly, it is usually kind of less of a pain to write directly assembly than to refactor and comment compiler assembly output.
  - camel-cdr 3y ago
    You botched the link. (I had the same problem btw, otherwise I would've included a godbolt link as well, but idk how to create one with noscript)