4 ms·
Does this in theory mean it should be relatively easy to port to coral TPU?
by syntaxing 4y ago
Does this in theory mean it should be relatively easy to port to coral TPU?
- saidinesh5 4y agoAll their tensor/math magic seems to happen in https://github.com/ggerganov/llama.cpp/blob/master/ggml.h https://github.com/ggerganov/llama.cpp/blob/master/ggml.h . So maybe if you implement the ggml.c with tensorflow/libcoral - you'd have a chance.
- sottol 4y agoAfaik that TPU has only 8MB of RAM to fit models, you'd have to continuously stream the weights - can't imagine that's workable.