jph00

23 ms·

301.
by jph00 3y ago
I think the model weights are the preferred form in which a programmer would modify the program. We modify models by fine-tuning them, which requires the weights, but not the training data/recipe.
302.
by jph00 3y ago
The board seems to be getting an awful lot of crap for not saying enough , and also getting crap because their initial letter was too direct rather than being the usual corporate mealy-mouth BS.
303.
by jph00 3y ago
I would have thought a non-profit board that puts the growth and success of their organization above the mission in their charter, even if they view the organization as going against their mission, sounds like the worst kind of board. How c
304.
by jph00 3y ago
FYI: one of the authors of this article, Karen Hao, just announced on Twitter that she's writing a book on OpenAI, and that this article is partly based on work done for that book.
305.
by jph00 3y ago
Helen Toner is brilliant. This dude has never met her, yet somehow has the gall to proclaim he'd be better at her job than he is. People shouldn't get board roles based on what degree they have, but based on how well they can do t
306.
by jph00 3y ago
Also, according to the article, the Ford is on road tires whilst the CT is on all-terrains.
307.
by jph00 3y ago
Minor correction: the article describes Chain of Density as "First introduced by Salesforce's AI Research wing" -- however the 1st author (who is a PhD student) and senior author are both at Columbia; only one of the 5 author
308.
by jph00 3y ago
It's pretty common to convert to bfloat16 nowadays since that's much faster. e.g. bitsnbytes now defaults to bf16 compute for 4bit quant.
309.
by jph00 3y ago
The thing I find most interesting about this language is that it's actually the original source of array/tensor broadcasting, which is now used pretty much everywhere (e.g. pytorch, numpy, tensorflow, etc...) https://tw
310.
by jph00 3y ago
tbh I think I'm more well known in AI than most of the people currently listed there, so I don't think that's the reason.
311.
by jph00 3y ago
It seems that the signatory form collection is not keeping up - I signed this 24 hours ago and my name still isn't listed. I suspect once they add the names that are coming in through the form that this will be a much longer list.
312.
by jph00 3y ago
It is not at all true that "AI is no different than proliferating nuclear weapons". The project manager for the Nuclear Information Project said ( https://www.vox.com/future-perfect/2023/7/3/23
313.
by jph00 3y ago
I was introduced to statistics with the mathematical approach, and it failed to help me understand why things were the way they were, due to the lack of context and application. Later I re-learned it in an applied way, and things made muc
314.
by jph00 3y ago
Lotus Improv actually predates Brio's product. I remember clearly when Improv came out -- it was a really exciting breakthrough at the time. But the big problem IMO was that it wasn't part of Lotus 123. So when Excel added pivotta
315.
by jph00 3y ago
Yeah I'm pretty sure it did - that website provides a service for just this kind of writing.
316.
by jph00 3y ago
I'm glad you like it Simon! And thank you for all the marvellous learning material you've provided. :D
317.
by jph00 3y ago
It makes a lot of sense! In fact there's a number of open source projects working on just such a model right now. Here's a great example: https://github.com/XueFuzhao/OpenMoE/
318.
by jph00 3y ago
I think that summary is pretty good, although it doesn't really highlight the most interesting bits, such as the fact that we implement a code interpreter from scratch, and that we cover fine-tuning to create a model that successfully
319.
by jph00 3y ago
Oh wow I only just uploaded this and it's on HN already! I'm actually quite excited about this video because I tried my hardest to pack all the key info I could think of into a 90 minute talk -- the goal is to be the one place I p
320.
by jph00 3y ago
ChatGPT can even show you how to do this in Python BTW! from ScriptingBridge import SBApplication notes_app = SBApplication.applicationWithBundleIdentifier_("com.apple.Notes") for folder in notes_app.folders(): for n
321.
by jph00 3y ago
awk and sed are cool, but whenever someone tells me they're interested in learning them, I always redirect them to learn perl's `-n` and `-p` flags instead, particularly with `-la` added. This gives you, basically, a superset of s
322.
by jph00 3y ago
IMO this 2016 article really hasn't aged well. It turned out that architectural improvements really did matter, and there was still loads of low-hanging fruit. His prediction that bayesian approaches (which were the topic of his PhD) w
323.
by jph00 3y ago
No I don't use the term overfitting for a model where the accuracy is getting better. I think it's misleading.
324.
by jph00 3y ago
We don't know. It's a report of some early experimental results. Our hope is that it will stimulate discussion and further research and development.
325.
by jph00 3y ago
If it was due to repetition there wouldn't be those sudden cliffs after each epoch.
326.
by jph00 3y ago
Cross entropy loss vs batch number
327.
by jph00 3y ago
Yes, a lot of recent research uses LLM outputs as training data, and it's been an extremely successful line of work.
328.
by jph00 3y ago
<deleted>
329.
by jph00 3y ago
As it happens I dug into this question in some detail a couple of weeks ago when analysing the dataset, including carefully reading the wikipedia page which the question comes from. AFAICT both D and E are kinda correct, but E isn't qu
330.
by jph00 3y ago
Oh wow yeah - I've also seen other people's training loss curves like that, going up during each epoch and then jumping down at the end of the epoch. I've never experienced that myself, and have no idea what's causing it

More ›