3 ms·
At what point will local inference catch up to today’s cloud inference? Will it ever? If it doesn’t, does that imply a certain dead-end for the LLM inference in
by datsci_est_2015 6mo ago
At what point will local inference catch up to today’s cloud inference? Will it ever? If it doesn’t, does that imply a certain dead-end for the LLM inference industry?
- lukaslalinsky 6mo agoI don't think at any point in foreseeable future we will have terabytes of RAM for dedicated LLM chips at home.