3 ms·
I am confused. He was hosting the 70B LLM everyone was demoing from his laptop? How can that serve the load? When I’ve run LLMs locally it is really taxing fo
by bhouston 2y ago
I am confused. He was hosting the 70B LLM everyone was demoing from his laptop? How can that serve the load? When I’ve run LLMs locally it is really taxing for just one concurrent session.