3 ms·
Confused about the title rewrite from “Fast and Portable Llama2 Inference on the Heterogeneous Edge” which more clearly communicates what this article is about
by jasonjmcghee 3y ago
Confused about the title rewrite from “Fast and Portable Llama2 Inference on the Heterogeneous Edge” which more clearly communicates what this article is about - a wasm version of llama.cpp.
I feel like editorializing to highlight the fact that it’s 2MB and runs on a mac are missing some of the core aspects of the project and write up.
- 3Sophons 3y agook.. should be run LLMs on my own devices with a 2MB portable app then?
- jaimex2 3y ago[flagged]
- doubloon 3y agowell it requires nvidia so maybe its not actually portable.
- threeseed 3y agoIt also works with Metal hence why they mention it runs on Mac.
- 3Sophons 3y ago'portable' in the article refers to the software's ability to run across various operating systems or environments, rather than its hardware dependencies? This means while the software can be installed and run on different OSs, certain hardware-specific optimizations (like those for Nvidia GPUs using CUDA) are necessary to achieve the best performance.
- PUSH_AX 3y agoNow I’m confused, because neither of the titles _clearly_ communicate that it’s a wasm version of llama.cpp in my opinion. It would probably be helpful to use the words “wasm” and “llama” to achieve that
- stavros 3y ago"Run LlaMA 2 on WASM in 2 MB RAM" This has the added advantage of being completely gibberish to someone outside tech. Edit: wait, it's not RAM, the binary is just 2 MB. That's disappointing.
- grumpy_tired 3y agoBeautiful. I wish all titles on hn where this concise.
- wongarsu 3y agoThe article is complete gibberish to someone outside tech, so if the role of the title is to describe the article to its intended audience yours is a lot better. Of course if you intend to communicate to non-tech people that you write relevant cutting-edge articles, then choosing a title like "Fast and Portable Llama2 Inference on the Heterogeneous Edge" does the job much better. Maybe even add the words sustainable and IoT somewhere.
- dang 3y agoThanks - a mod replaced the title last night. (Submitted title was "Run LLMs on my own Mac fast and efficient! Only 2 MBs.") Submitters: "Please use the original title, unless it is misleading or linkbait; don't editorialize." - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html