jph00

13 ms·

211.
by jph00 2y ago
Pretty much all models, including today's models, already fall foul of the "Hazardous capability" clause. These models can be used to craft persuasive emails or blog posts, analyse code for security problems, and so forth. Wh
212.
by jph00 2y ago
I've written a submission to the authors of this bill, and made it publicly available here: https://www.answer.ai/posts/2024-04-29-sb1047.html The EFF have also prepared a submission: https://www.contex
213.
Call-to-Action on SB 1047 – Frontier Artificial Intelligence Models Act(affuture.org)
148 points by jph00 2y ago|103 comments
214.
by jph00 2y ago
Weights will be realised tomorrow, according to one of the tech report authors on Twitter.
215.
by jph00 2y ago
Jeremy from Answer.AI here. Let me know if you have any questions or comments about this work. (Although I can't take any credit for it -- this is the work of Kerem Turgutlu!)
216.
Efficient finetuning of Llama 3 with FSDP QDoRA(answer.ai)
13 points by jph00 2y ago|3 comments
217.
by jph00 2y ago
FYI, for anyone interesting in creating and using hooks to better understand what's happening in your model, I created a free lesson covering that: https://course.fast.ai/Lessons/lesson17.html
218.
by jph00 2y ago
The OSI definition applies to source code -- I'm not sure the term "open source" makes much sense applied to model weights. Whilst I agree the term isn't ideal, I don't agree with the other comments in the post I or
219.
by jph00 2y ago
> " Nothing about Meta's license is open source. It's a carefully constructed legal agreement intended to prevent any meaningful encroachment by anyone, ever, into any potential Meta profit, and to disavow liability to pre
220.
by jph00 2y ago
Looks nice, but url2text doesn't seem to have an API, and urlbox doesn't seem to have an option to skip the screenshot if you only want the text. And for just the text, it looks to be really expensive.
221.
by jph00 3y ago
Why is only one of them on WebGPU? Is it because there additional tricky steps required to make a model work on WebGPU, or is there a limitation on what ops are supported there? I'm keen to do more stuff with WebGPU, so very interested
222.
by jph00 3y ago
There's some strong baselines for CNNs here that would be great to compare to. https://github.com/fastai/imagenette
223.
by jph00 3y ago
Miqu is a leaked model -- no license is provided to use it. Yi 34B doesn't allow commercial use. Deepseed 33B isn't much good at stuff outside of coding. So it's fair to say that DBRX is the leading general purpose model that
224.
by jph00 3y ago
I fail to see how a "quick perusal" could possibly be sufficient to come to such a sweeping conclusion.
225.
by jph00 3y ago
This PR is super-helpful -- it's nearly impossible for me to work with projects that pin to specific versions, since it leads to so many conflicts. Having minimum versions like in this PR makes life much easier!
226.
by jph00 3y ago
Doesn't Anvil force you to run stuff on their proprietary service however -- which costs money? (I'm not saying that's not a good option for some people, but it's very different to an open source framework you can run an
227.
by jph00 3y ago
All programming is abstractions, unless you're using machine code. Abstractions don't necessarily make debugging harder -- done the right way, they can make it easier. (Yes, you can call it "pure python" if it compiles t
228.
by jph00 3y ago
I really like this idea of using Python to create both the frontend and backend. Another lib doing this is https://solara.dev/ . Something I particularly like about Solara is that you can interactively build your app in a J
229.
by jph00 3y ago
He thinks it's really amazingly cool! :D I'm so happy to hear that I had some part to play in inspiring such a marvellous project.
230.
by jph00 3y ago
NVlink does not appear as unified memory. But NVlink + two 3090 does work great with FSDP/QLoRA (I have just such a machine at home!)
231.
by jph00 3y ago
Fair -- 24GB GPUs are still quite expensive. >10x cheaper than 80GB GPUs though! We already have something we're testing that's gotten 70b training down to dual 16GB GPUs, and we're working it making it even smaller...
232.
by jph00 3y ago
Yes you can use multiple computers in parallel with this script. And there's lots of GPU providers out there (the article links to an aggregator of providers and prices).
233.
by jph00 3y ago
It's an interesting question. I'm not sure we really know yet what the right mix of RAG and fine tuning is. IMO small-scale fine tuning might be under-appreciated.
234.
by jph00 3y ago
That should work great, although we haven't tried it. Let us know what you find!
235.
by jph00 3y ago
Yes, you can run FSDP/QLoRA over multi-node. There's a slurm script in the repo showing how to do it.
236.
by jph00 3y ago
Yes I'm familiar with Teknium's Mistral issues, which were resolved some time ago. IIRC they weren't related to FSDP.
237.
by jph00 3y ago
Good news -- axolotl has just merged support for FSDP/QLoRA training, thanks to a rapid collaboration between the axolotl and Answr.AI teams!
238.
by jph00 3y ago
As mentioned in the post, benchmarking results are coming in a later post. But in short: you can train an epoch of Alpaca in 24 hours or so, which is enough to get very significant change in model behavior.
239.
by jph00 3y ago
The title of that podcast isn't something I actually said (IIRC). I commented in that interview that I feel we should not consider pre-training and fine-tuning to be as separate as we do now.
240.
by jph00 3y ago
I believe my answer is accurate. I don't know on what basis you claim otherwise. This is my work, and I'm well placed to make the assessment. My response is based on reading a great many papers and doing many experiments over the

More ›