3 ms·
"The recent change also means you can run multiple LLaMA ./main processes at the same time, and they'll all share the same memory resources." So this could have
by abhimskywalker 4y ago
"The recent change also means you can run multiple LLaMA ./main processes at the same time, and they'll all share the same memory resources."
So this could have a main and multiple sub-worker llm processes possibly collaborating while sharing same memory footprint?
- l33tman 4y agoYes, if the model is mmap'ed read-only (as I'm sure it is). There are other bottlenecks than CPU cores though, it might not be very useful to run multiple in parallel..