4 ms·
There's a huge degree of data re-use in the weights. This should be exploited. Second, one could envision paging the hidden units back to system memory on a c
by varelse 11y ago
There's a huge degree of data re-use in the weights. This should be exploited.
Second, one could envision paging the hidden units back to system memory on a coprocessor-based implementation (GPUs/FPGAs/not Xeon Phi, gag me). 256 GB servers are effectively peanuts these days relative to developer salaries and university grants (datapoint: my grad school work system was ~$100K in 1990 dollars) so unless you're trying to create the first strong AI, I don't think this is a serious constraint.
Good luck with that no matter what Stephen Hawking, Elon Musk, and Nick Bostrom harp on about: we have no idea what the error function for strong AI ought to be and even if we did, it's over a MW using current technology to achieve the estimated FLOPS of a human cerebrum.
- tormeh 11y agoI meant that the state vector has constant size and just setting it at the maximum available might give you problems with training.
- varelse 11y agoNothing you can't work around if you're willing to roll your own code. That said, I agree 100% if you're dependent on someone else's framework...
- tormeh 11y agoI meant that the state vector has constant size and just setting it at the maximum available might give you problems with training.