4 ms·
Neutrino-1 8B
- deleted 3mo ago[deleted]
- yborg 3mo agoLargely outperformed by Ternary-Bonsai-8B by their own chart, doesn't seem clear what their special sauce is here.
- kamranjon 3mo agoThere's a really interesting trend of labs using "proprietary" methods to convert existing models to a compressed ternary format. PrismML actually targeted the same Qwen 8b model and got it down to 1.75gb here: https://prismml.com/news/ternary-bonsai https://prismml.com/news/ternary-bonsai I wonder how proprietary it all is though, since the BitNet b1.58 paper has been out for a couple years now: https://arxiv.org/abs/2402.17764 https://arxiv.org/abs/2402.17764 From the wikipedia on 1.58 bit llms: "BitNet derives its performance from being trained natively in 1.58 bit instead of being quantized from a full-precision model after training. Still, training is an expensive process, and it would be desirable to be able to somehow convert an existing model to 1.58 bits. In 2024, HuggingFace reported a way to gradually ramp up the 1.58-bit quantization in fine-tuning an existing model down to 1.58 bits." The section from huggingface is here: https://huggingface.co/blog/1_58_llm_extreme_quantization#fine-tuning-in-158bit https://huggingface.co/blog/1_58_llm_extreme_quantization#fi... I just wonder how many of these labs are basically following the huggingface recipe here and possibly tweaking it and releasing models without huge training costs.
- Twirrim 3mo agoIndependent testing of prismml suggest quite a capability drop off outside of their cherry picked benchmarks. I'll be curious to see what this model achieves though.
- kamranjon 3mo agoUnfortunately Fermion Research appears to entirely AI generate all of their content here, even for the research section: https://www.fermionresearch.com/research/neutrino-8b/ https://www.fermionresearch.com/research/neutrino-8b/ "Neutrino-1 8B was trained natively in its shipping format. There is no full-precision product model that was rounded afterward: the ternary representation is the medium the weights learned in, and the training methods that hold this quality at this depth are the lab’s unpublished work. The findings below are the part that travels." This statement seems misleading at best. Both the model page and the release page are basically unintelligible - I don't have a ton of faith in the work here, at least PrismML write coherent releases for their models. Edit: Another beautiful piece of prose here, I almost wonder if they used the 8b model to generate the content for this release... "Across the 6.95B coded weights, 62.63% sit at zero and the remainder splits 18.68% plus to 18.69% minus: sign-balanced to a hundredth of a point with no constraint asking for it."
- LtdJorge 3mo agoI guess it's saying how many of the weights are -1, 0 or +1.
- jdiff 3mo agoIt is, but why? And what's with the bizarre way of phrasing that? Why the bizarre observation that, indeed, nobody asked for it?
- rcxdude 3mo agoAlso, wouldn't normalisation tend to produce a mean of zero in the weights, anyhow?
- mapt 3mo ago> sign-balanced to a hundredth of a point with no constraint asking for it." This isn't their model, this is (probably?) ChatGPT doing a brag / promo deck authorial voice. It routinely uses half a dozen sentence constructions that are relatively uncommon in normal or technical speech. Eccentric. Persuasive. Trying too hard. Restating its point in a promotional way that doesn't sound natural, leading into a sentence where it hyperbolically sells you on having done the impossible. Normal persuasive speech uses these constructions, especially public speaking doing a VC pitch or an Ancient Aliens, but it would be fucking strange if a person started chaining them in normal conversation and using little else, trying to insinuate competence. You start to recognize it pretty quickly on Youtube.
- deleted 3mo ago[deleted]
- moinism 3mo agoThe content on that page is too AI-generated to make sense to me; I don't understand what the model is for.
- SwellJoe 3mo agoAI doesn't want anything, so it doesn't care whether it conveys meaning in its writing. And, apparently the developers of this project also don't care whether it conveys meaning. They just assume we'll wade through the slop? I dunno.
- iwontberude 3mo agoIt has no ability to ground output in causality. It does not understand what causes humans to learn. It’s a correlation and randomness engine.
- naruhodo 3mo agoI have the same question. I get that it’s designed to run on a CPU, big GPU or MacBook (although the way that was phrased confused me at first). I’m struggling with what a “decoder-only” model is good for.
- madhu_ghalame 3mo ago[dead]
- NetOpWibby 3mo agoThere’s a new announcement every other day wrt models. How do y’all keep track of them all same know what’s decent? Good grief! And if it’s decent today, it’s shit in eight months! I tool hop as much as the next dev but this is a bit much.
- weikju 3mo agoI just tune out. It’s not worth knowing, following every development in the field. If something works now it will probably work in 8 months even if it’s no longer the new hype thing. Who cares. Not using any of it is also a valid option though it doesn’t satisfy your FOMO. But nothing ever will.
- badatnames 3mo agoThere's no need to, the 50 foot view is simply that many alternatives exist and they mostly fall into 3 meaningful weight classes with comparable performance among each class's members: too expensive to use indiscriminately, too big to run at home, and too small for complex work. As for names and faces in between, the overarching conclusion is that we're rapidly approaching commodity status and those don't really matter much
- cyanydeez 3mo agoI'm working on solidifying local LLM work flow so I can stop tweaking the nobs and build things that arn't AI related. There's tons of people caught up in the AI gold rush, but the value of real products for real people is out there. Even more so when you exclude the "I slapped AI onto this"
- runtime_lens 3mo ago[flagged]
- Havoc 3mo agoCan’t say I’m a fan of containers for this. A big chunk of local LLM gains come (imo) from the open modular nature of llama.cpp and friends. Easy to modify. Easy to experiment. Containers are the proprietary binary blob in hardware world equivalent
- embedding-shape 3mo agoWhat? How are those even related with each other? You can just as easy modify and experiment with llama.cpp in a container as outside of it, they really shouldn't impact one another. Containers don't suddenly make llama.cpp less "open modular" somehow, and I'm not sure how you'd arrive as such conclusion.
- Alien1Being 3mo agoAI slop site with AI slop research... Blog populated with incoherent PR material generated by Yet Another AI. Sigh...
- zoom6628 3mo agoCan Dang implement a slop rating on submitted pages? Not a block but at least a % likelihood of AI slop content and that could also be tied with a BS rating as well. Could use AI for both which seems hilariously appropriate.
- codeduck 3mo agoOr perhaps a slop flag on submissions that gets added once enough readers have flagged it as such. I'd love that feature.
- secult 3mo agoThere is not a single person mentioned on the website, github created 3 days ago, no real contact, everything hidden. Completely anonymous. Domain owner hidden.
- drbscl 3mo agoSlop article, slop site... slop model?
- sparse-Matrix 3mo agoUnfortunately it crashed out 'no space left on device' while installing the python demo/quickstart. Only problem was there is plenty of space on the device. PLENTY (not quite 750gb).
- wincondition 3mo agoare you pn linux? reproduced the same thing just now its pip dumping torch into /tmp while it installs, and on most distros /tmp is ram bound and capped at half your memory, so it fills up while your actual disk jsut sits there. export TMPDIR=/some/big/path before installing should fix it. adding a proper check next release so it actually tells you which folder ran out.
- bmiekre 3mo agoThis all sounds like middle-out
- codeduck 3mo agoSo... not a new subatomic particle discovery then.
- androiddrew 3mo agoSo I'd like to see a Nemotron3 Ultra converted to a 1.58bit format then have them retrain on the open dataset.
- wincondition 3mo agohey guys, founder here. I am working completely solo right now running on grant funding, so things slipped through on the website quality, it is largely written by AI, I'll deslopify the blogs and be much clearer about methodology. Neutrino-1 is a starting point for Fermion, It's a density experiment on how much capability survives per byte. rn working on really fast and efficient TTS and realtime models to which this architecture is generalizing pretty well. Sorry for all the confusion from reading through the blogs, should be fixed in a couple of days!
- deivid 3mo agoFor me, sloppy content is worse than no content. I'd rather have 3 sentences written by a human than a wall of Claudespeak. Do you have an RSS or a way to subscribe to the TTS work you are doing? Your newsletter button is just a mailto link
- JSR_FDED 3mo agoLots of respect for a solo founder. But you submitted this yourself, and it’s incomprehensible.
- closetothe 3mo agoI was really hoping this would have something to do with neutrinos
- pavel_lishin 3mo agoDamn, I was hoping this was about an actual neutrino.
- supjeff 3mo agosoftware naming claims another victim
- thefrench_theod 3mo ago[flagged]