3 ms·
General hardware. I used to do all my work on dedicated hardware with support for Lisp (CADRs, 36xxs, D-machines, and before that PDP-10s) because those genera
by gumby 8y ago
General hardware. I used to do all my work on dedicated hardware with support for Lisp (CADRs, 36xxs, D-machines, and before that PDP-10s) because those general purpose machines couldn't implement important features like generational garbage collection without specific hardware assist.
Then one day in 84/85 I saw a generational GC using the MMU on a 68K and the light dawned.
Thus while I'm pretty interested in ML chipsets, I figure the necessary portion of functionality will ultimately end up subsumed in the generic chips and most if not all those ML hardware startups will go bust.
- chalst 8y agoI wouldn't bet on the ML hardware startups either, but don't you think that what we regard as general hardware is always going to be too heavyweight to be suitable for massive parallelisation?
- gumby 8y agoWell, take a system approach to the hardware. It's possible you could have multiple paths to memory. It's more likely you won't on non-high-end-systems. The primary path will for the CPU, especially on mobile. Also look at the workload: often many of the primary cores will be idle when you're crunching a big dataset. Those two factors suggest that you want to use CPU hardware for mult-add fops, especially when you consider using smaller float sizes (not just f16 but even f8). And then you consider that the compiler can perhaps properly interleave these ops with the regular instruction mix... This is who general purpose von Neumann architectures ate hardware, both RISC and CISC. Now the current crop is pretty dire (e.g. AVX* is hard to use with a typical workload) but perhaps these won't be typical workload? Or the power/timing problems will be solved other ways.
- comex 8y agoPerhaps. But before ML was on dedicated hardware it was on GPUs, and programmable GPUs have been around for almost two decades now. They're getting more and more CPU-like, but so far there's no sign of them actually being subsumed by CPUs. If anything, attempts to make CPU-GPU hybrids have historically tended to fail, such as Larrabee or the Cell processor on PlayStation 3. Of course, ML is a somewhat different workload from the usual use of GPUs... but not that different. The future could be different from the past... but as CPU speed increases get slower and slower, why not expect things to shift even further in the direction of dedicated hardware?
- lispm 8y agoI saw the better GCs were implemented for machines with the 68851 MMU. Later the 68030 had the necessary MMU functionality built in. Both allowed for example Macintosh Common Lisp to have an ephemeral GC, IIRC.