jph00

19 ms·

241.
by jph00 3y ago
Yes, sparsification is another useful approach for higher efficiency, although block sparse kernels are pretty complex to work with -- especially when combined with quantization and LoRA! Most of the sparsity papers I've seen use &quot
242.
by jph00 3y ago
DeepSpeed has more features than FSDP, but it's much more complex to hack on -- FSDP is written directly in python using calls to the PyTorch library, whereas DeepSpeed is 20% C++ and 10% CUDA (according to the GitHub stats). We'v
243.
by jph00 3y ago
The article actually sneaks in a footnote that answers this ( https://www.answer.ai/posts/2024-03-06-fsdp-qlora.html#fn1 ): "Throughout this article “training” can refer to either pre-training, or fine-tuning".
244.
by jph00 3y ago
Diffusion unet has an "extended" version nowadays that applies to the resnet part as well as the cross-attention: https://github.com/cloneofsimo/lora
245.
by jph00 3y ago
One thing I forgot to mention in the post which I think is kinda cool: at the NeurIPS Efficiency Challenge this year, where Tim Dettmers and I both did keynotes, every single top-ranked entry used QLoRA! The challenge was to create the most
246.
by jph00 3y ago
Yes, along with the many other <4 bit quant methods recently developed -- there's been a wonderful boom in low-bit quant methods in the last 6 months, and we've got our own ideas for taking them further too. Along with QLoRA&#x
247.
by jph00 3y ago
For now, it's for finetuning. The issue of to what degree it might be possible to train a model from scratch using QLoRA is still an open question. The relora paper showed that it can work in some situations, but attempts to scale it u
248.
Fine tune a 70B language model at home(answer.ai)
909 points by jph00 3y ago|206 comments
249.
by jph00 3y ago
> assembly is comparatively easier than, say, formal logic or partial differential equations Can you explain how, exactly, one compares the ease of assembly to the ease of formal logic, such that one can then make a definitive and swee
250.
by jph00 3y ago
> we expect AI to get there first because it's denser and can be replicated orders of magnitude more quickly and cheaply There is no evidence that AI that's useful enough to help make research breakthroughs can be replicated
251.
by jph00 3y ago
There is only a single mention of anything even vaguely like what you're reacting to, which is the specific claim of an "intentionally faulty calculation as part of the computational RoboDebt program" in Australia. To jump fr
252.
by jph00 3y ago
Rachel and I are well aware of the promise of AI as a research tool. I created the first company that focussed on deep learning in medicine. Rachel has a PhD in math and is now doing a masters in immunology, and is trying to help figure out
253.
by jph00 3y ago
Please folks, do take the time to submit a carefully written comment to this, because if legislation is introduced to limit or ban open models, then it's going to cause huge pain for society at large -- possibly as far as undoing the v
254.
by jph00 3y ago
These comments appear to be about Gemini's image generation, IIUC. Gemma, however, is a language model -- whilst I believe that a larger unreleased version of it is used as part of the Gemini product, it doesn't seem relevant to t
255.
by jph00 3y ago
vLLM uses transformers.
256.
by jph00 3y ago
When I was at FastMail I did a lot of very manual work to not just block spammers and other abusers, but to make their life as difficult as possible. That included figuring out how to notify the people running the servers they used (inclu
257.
by jph00 3y ago
Yeah I've never even heard of this author before, but this response is so nicely written and well considered that I'll be checking out their books now!
258.
by jph00 3y ago
But hasn't the concentration you're describing been created under the current copyright system? Those winning from this system appear to be exactly those groups that are concentrating power the way you describe.
259.
by jph00 3y ago
Python has a GC.
260.
by jph00 3y ago
So excited to see this coming! When we used Jupyter Notebooks to publish Practical Deep Learning for Coders, we discussed with Allen the idea of doing the same for some of his books. Now it's actually happening. :D Even better, he&#x27
261.
by jph00 3y ago
To clarify -- I don't mean this as a criticism. If HN wants to censor swearing, that's fine; but it probably should be mentioned in the guidelines in that case.
262.
by jph00 3y ago
HN guidelines: > If the title contains a gratuitous number or number + adjective, we'd appreciate it if you'd crop it. E.g. translate "10 Ways To Do X" to "How To Do X," and "14 Amazing Ys" to &qu
263.
by jph00 3y ago
I think he's actually more towards the end of "actually open", which is not a place either Elon nor Sam are at. Grok and OpenAI don't openly publish or freely release much of their work. Andrej however has released a lot
264.
by jph00 3y ago
You're reacting to the style and medium of the post, and the handle and race of the author. Maybe consider looking at the science too? (FWIW, I've started working on AI for radiology nearly a decade ago, and Chinese researchers ha
265.
by jph00 3y ago
> We often get mired in the debate of "should people be able to learn to make bombs or how to best kill themselves" (and those are important debates), but our solutions to those problems will have big implications for future a
266.
Self-Discover: Large Language Models Self-Compose Reasoning Structures(arxiv.org)
4 points by jph00 3y ago|0 comments
267.
by jph00 3y ago
I'm not really trying to rebut Michael's argument -- I think it's true, to an extent, some of the time. But I think it's more true more of the time in the reverse direction. So I don't think it's a good argumen
268.
by jph00 3y ago
Thank you Michael. I'm not even sure I disagree with you on many things -- I think things are very complicated and nuanced and am skeptical of people that hold overly strong opinions about such things, so I try not to be such a person
269.
by jph00 3y ago
There are far far more dollars available to people that are on the "AI Safety" bandwagon than to those pushing back against it. The idea that the Upton Sinclair effect is the source of pushback against AI Safety zealotry, is get
270.
by jph00 3y ago
In the US there’s another option: become a Public Benefit Corporation (PBC). That’s what we (Answer.AI) did. It provides a great solution to exactly this problem: you don’t have to sell your company just because the financial offer is high;

More ›