4 ms·
Hang on, how unlimited is unlimited here? Surely the immediate thing you'd do with this is just never delete any prior inputs so it becomes defacto long term me
by XorNot 4y ago
Hang on, how unlimited is unlimited here? Surely the immediate thing you'd do with this is just never delete any prior inputs so it becomes defacto long term memory for the model?
- shishy 4y agoLast paragraph touches on that: The length of inputs is theoretically bounded by the memory limitations of the computer used. More practically, using a CPU datastore is many times slower than a GPU datastore because of slower search and the need to transfer retrieved embed- dings to the GPU... (continues)
- 0xDEF 4y agoThe limit is RAM but GPU RAM is much faster than computer RAM.
- davrosthedalek 4y agoIs that really the limit? There is no real restriction that everything is in memory at the same time, right? You could maybe stream from SSD?
- capableweb 4y agoCreate a swapfile and you essentially trade disk space for memory space.