3 ms·

Trivial Artificial Neural Network in Assembly Language

51 points by fogus 15y ago

kruhft 15y ago
I don't think I had more fun and felt more control than when I was professionally programming in assembly language. The more I use high level languages the more I just want to step back and learn x86-64 assembler and put scoping and default allocation and language lawyers behind me. Of course there's nothing stopping me except myself really...
losethos 15y ago
LoseThos started 100% TASM. Then, as I improved my compiler, I converted it to C. I have many years of professional asm experience. The main reason it's bad is when you want to modify a routine to add something, suddenly you have to change all the registers. You know how Mexicans have machizmo? Asm is generally considered manly... like guns and weapons. The most retarded thing are OSes that are all asm, unless it's because they have a native assembler and not compiler. MikeOS and BareMetal? All asm is just silly. Here's LoseThos when it was all TASM. http://web.archive.org/web/20040606212724/http://www.simstructure.hare.com/OS.htm http://web.archive.org/web/20040606212724/http://www.simstru... It's a slow link. Fixation on asm is childish -- you need to do it to your hearts content, then you will not be emotional. Actually, my niche is supposed to be recreational programming. ASM is popular and it's sure not because it warranted -- it's highly entertaining.
xcallmejudasx 15y ago
How long did it take you to put this together and how much of that time was coding vs researching?
dave_sullivan 15y ago
That's really cool, a good breakdown on what's actually going on with an ann. But I don't think using assembly will help with speed, wouldn't you still be better off using matrix multiplication on a gpu, written with something like theano in python? Then again, maybe you just used assembly to explain things at a very low level, rather than for any speed boost. Either way, very cool article.
- pbsd 15y ago
  In terms of performance, it's actively worse than a compiler's output. Using the x87 FPU makes no sense in this day and age, considering this is Intel 64 assembly. GPUs (and plain CPU SIMD) would help very much yes. Any serious NN implementation will be using it.
  - pslam 15y ago
    It's not only worse, it's poorly scheduled. There are many times hand-coding assembly will do far better than a compiler (often an order of magnitude). This isn't one. If you write straight-line asm code where every output is immediately consumed in the next 2-3 instructions, you'll find it runs exactly the same speed as the compiler's output. Probably worse, because the compiler would at least attempt this. This is where a human can do better than the compiler, but this isn't an example of it.
    - pbsd 15y ago
      Well, I give scheduling a free pass because modern processors will reorder and rename the whole thing anyway, provided the instructions are not too badly chosen. Beating the compiler usually has more to do with having more information. Say, "this register will never have integers greater than X", or "the carry flag will be set to Y here", or whatever. Even compiled intrinsic SIMD beats most hand-coded kernels these days, unless you're doing something very exotic (e.g., mixing integer and floating-point logic to save cycles). Hand-written assembly is something that will probably go away nearly completely (can't really beat optimal schedulers, superoptimizers), and I say this as someone who's spent many years optimizing assembly.
      - pslam 15y ago
        As someone who has spent many years working with different cores and in the design of them, static, hand-optimization is never going away. Dynamic optimization by the core involves deeper pipelines and/or speculative execution and/or many more gates, all of which takes up more area and power. You end up spending more energy than you save, and pretty much everything is power consumption (or cooling, which is essentially the same thing) limited these days. Super-optimization is intractable for large sequences. It's like trying to solve a chess game by brute force. Likewise for other compile-time optimizations - they're polynomial or worse. You at least need a human to design the structures with knowledge of how they're represented low level and how they'll end up being pumped. It needs someone to come up with better compiler algorithms before we can say for sure that hand-optimization will go away. Increase in CPU speed alone doesn't make it inevitable.

51 points by fogus 15y ago

I don't think I had more fun and felt more control than when I was professionally programming in assembly language. The more I use high level languages the more I just want to step back and learn x86-64 assembler and put scoping and default allocation and language lawyers behind me. Of course there's nothing stopping me except myself really...

losethos 15y ago

LoseThos started 100% TASM. Then, as I improved my compiler, I converted it to C. I have many years of professional asm experience. The main reason it's bad is when you want to modify a routine to add something, suddenly you have to change all the registers. You know how Mexicans have machizmo? Asm is generally considered manly... like guns and weapons. The most retarded thing are OSes that are all asm, unless it's because they have a native assembler and not compiler. MikeOS and BareMetal? All asm is just silly. Here's LoseThos when it was all TASM. http://web.archive.org/web/20040606212724/http://www.simstructure.hare.com/OS.htm http://web.archive.org/web/20040606212724/http://www.simstru... It's a slow link. Fixation on asm is childish -- you need to do it to your hearts content, then you will not be emotional. Actually, my niche is supposed to be recreational programming. ASM is popular and it's sure not because it warranted -- it's highly entertaining.

xcallmejudasx 15y ago

How long did it take you to put this together and how much of that time was coding vs researching?

dave_sullivan 15y ago

That's really cool, a good breakdown on what's actually going on with an ann. But I don't think using assembly will help with speed, wouldn't you still be better off using matrix multiplication on a gpu, written with something like theano in python? Then again, maybe you just used assembly to explain things at a very low level, rather than for any speed boost. Either way, very cool article.

pbsd 15y ago
In terms of performance, it's actively worse than a compiler's output. Using the x87 FPU makes no sense in this day and age, considering this is Intel 64 assembly. GPUs (and plain CPU SIMD) would help very much yes. Any serious NN implementation will be using it.
- pslam 15y ago
  It's not only worse, it's poorly scheduled. There are many times hand-coding assembly will do far better than a compiler (often an order of magnitude). This isn't one. If you write straight-line asm code where every output is immediately consumed in the next 2-3 instructions, you'll find it runs exactly the same speed as the compiler's output. Probably worse, because the compiler would at least attempt this. This is where a human can do better than the compiler, but this isn't an example of it.
  - pbsd 15y ago
    Well, I give scheduling a free pass because modern processors will reorder and rename the whole thing anyway, provided the instructions are not too badly chosen. Beating the compiler usually has more to do with having more information. Say, "this register will never have integers greater than X", or "the carry flag will be set to Y here", or whatever. Even compiled intrinsic SIMD beats most hand-coded kernels these days, unless you're doing something very exotic (e.g., mixing integer and floating-point logic to save cycles). Hand-written assembly is something that will probably go away nearly completely (can't really beat optimal schedulers, superoptimizers), and I say this as someone who's spent many years optimizing assembly.
    - pslam 15y ago
      As someone who has spent many years working with different cores and in the design of them, static, hand-optimization is never going away. Dynamic optimization by the core involves deeper pipelines and/or speculative execution and/or many more gates, all of which takes up more area and power. You end up spending more energy than you save, and pretty much everything is power consumption (or cooling, which is essentially the same thing) limited these days. Super-optimization is intractable for large sequences. It's like trying to solve a chess game by brute force. Likewise for other compile-time optimizations - they're polynomial or worse. You at least need a human to design the structures with knowledge of how they're represented low level and how they'll end up being pumped. It needs someone to come up with better compiler algorithms before we can say for sure that hand-optimization will go away. Increase in CPU speed alone doesn't make it inevitable.