imjonse

16 ms·

271.
by imjonse 3y ago
It may not be harmful overall, that is the point. You can get people to eat less of the stuff that makes them ill without requiring them to have time for or interest in science and details. Anyone with enough energy to care will pick the su
272.
by imjonse 3y ago
What percentage of cured meat and cold cuts consumed worldwide (or in your locale) would you say are modern vs. traditional?
273.
by imjonse 3y ago
How was it a surprise with such a name?
274.
by imjonse 3y ago
You need to find a middle ground between a high level and easily labeled category and scientific rigor. If the aim is raising awareness in consumers, erring on the side of generality and dumbing down is preferred.
275.
by imjonse 3y ago
737 Max whistleblowers should take notice though.
276.
by imjonse 3y ago
Attack here is a term specific to cryptography and infosec, it does not imply doing anything illegal or violent. "Oracle attack" is the more specific term where you probe a system to give up some of its internals it was not meant
277.
by imjonse 3y ago
"Reverse engineering part of a production language model" has a more boring ring to it though.
278.
by imjonse 3y ago
Tell me which of the large problems solved by YC companies - including those with 8-9 figures of investment rounds not just the initial seed the 600k compares to - can be likened to the challenges of climate action. There are the ones doing
279.
by imjonse 3y ago
It has a good treatment of recurrent neural networks, which is nice since the theory behind them starts being relevant again. For comparison, both Bishop's new deep learning book and Simon Price's Understanding deep learning get s
280.
by imjonse 3y ago
It is not written for application programmers, it is a machine learning book.
281.
by imjonse 3y ago
Effective climate action will need several orders of magnitude more money than 600K :) However well-meaning, cute high-school projects and rural third world ingenuity is not going to cut it. This is a problem best addressed by Manhattan&#x2
282.
by imjonse 3y ago
Not new, launched in 2021 apparently.
283.
by imjonse 3y ago
> To some extent, they may be right that open sourcing AGI would lead to too much danger. They claimed that about GPT-2 and used the claim to delay its release.
284.
by imjonse 3y ago
From the blog's footnote: "In addition, we’d like to note that engineers have worked to optimize prompts and few-shot samples for evaluations and reported higher scores for a newer GPT-4T model"
285.
by imjonse 3y ago
Meanwhile there is the 'Foundation model transparency index' from Stanford https://crfm.stanford.edu/fmti/
286.
by imjonse 3y ago
Is the bright body visible from 0:25 to 0:35 the Sun? It looks much brighter already at 4:04.
287.
by imjonse 3y ago
The term open source fell prey to hype, ignorance and VC money in the case of AI models. The most incorrect uses are for literally binary blobs without a recipe to reproduce (no training data in the majority of cases) and for products that
288.
by imjonse 3y ago
Nothing there yet, but it's good to know they want to publish just did not get around to yet.
289.
by imjonse 3y ago
You're both right, I skimmed the paper, saw large model numbers but didn't notice it was for speed. On the HF page they say those models are being trained. https://huggingface.co/papers/2402.17764 "We ha
290.
by imjonse 3y ago
The presentation is simplified because it implies knowledge of its predeccesor, BitNet https://arxiv.org/abs/2310.11453
291.
by imjonse 3y ago
They present numbers for 7B to 70B models as well.
292.
by imjonse 3y ago
Too bad there seem to be no pretrained models to download. This is not a quantization method to apply on existing models, so having the pretrained weights is needed if one wants to test it.
293.
by imjonse 3y ago
That is normal in companies. I am more worried by the cases of 'I educated myself by using public funds for my research at uni, then went of to create a company and privatize the profits'
294.
by imjonse 3y ago
search for 'Mixtral 8x7B' in the text
295.
by imjonse 3y ago
The FFT is present in some of the pre-Mamba state space models as well to optimize applying the convolution.
296.
by imjonse 3y ago
Explaining Mamba is a rite of passage, like the monad tutorials of yore.
297.
by imjonse 3y ago
Most (all?) open-ish 7B+ models today are finetunes of proprietary/semi-closed/bigbudget LLMs. There is no such foundation model for Mamba yet.
298.
by imjonse 3y ago
Low adoption is primarily caused by it being relatively recent and there are no 7B or larger public Mamba-based models to start a comparison in earnest with widely used transformer based LLMs.
299.
by imjonse 3y ago
how about models + a tailwind SaaS boilerplate? /s
300.
by imjonse 3y ago
Also in that article he says "In fact, I’d go as far as to say that The concept of attention is the most interesting recent architectural innovation in neural networks." when the initial attention paper was less than a ye

More ›