Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
leiframming
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
3 ms
·
1.
▲
by
leiframming
3y ago
No, I think the author gets this mixed up. Zero copy does not play a role here - the problem is how the model is being loaded in the steps that are shown in the article and they have nothing to do with Safe tensors' zero copy ability.
2.
▲
by
leiframming
3y ago
Interesting article, but the premise is wrong. You don't need 720GB ram to load the model. The author claims that you need to first load up the model (360GB) and then the load the weights (another 360 GB) and the replace the first with