7 ms·
Try MLC-LLM. Its not as bad as you'd think. In the future(?), they will probably use the AI blocks instead of the GPU, which are very low power.
by brucethemoose2 3y ago
Try MLC-LLM. Its not as bad as you'd think.
In the future(?), they will probably use the AI blocks instead of the GPU, which are very low power.