zxexz

10 ms·

121.
by zxexz 2y ago
Yeah, people will go crazy for GPT-o2 trained on the readings of sensors "barely embedded" in the brains tortured monkeys, for sure. EDIT: This comment may have been a bit too sassy. I get the thought behind the original comment,
122.
by zxexz 2y ago
I can't help but feel that this talk was a lot of...fluff? The synopsis, as far as my tired brain can remember: - Here's a brief summary of the last 10 years - We're reaching the limit of our scaling laws, because we've
123.
by zxexz 2y ago
Really not sure what you mean by this, could you explain?
124.
Byte Latent Transformer: Patches Scale Better Than Tokens(ai.meta.com)
378 points by zxexz 2y ago|84 comments
125.
by zxexz 2y ago
FWIW, I'm in a jurisdiction with a 10 year retention requirement, and also have had no issues getting imaging data from beyond that (on DVD, of course - one department even had a side hustle selling external DVD drives for $50 on the s
126.
Sycamore Partners Is in Talks to Acquire Walgreens(finance.yahoo.com)
1 points by zxexz 2y ago|0 comments
127.
by zxexz 2y ago
Yes, they use a pre-trained model, but they do further training (please correct me if I mis-read, and also I realize my above comment could be interpreted as saying they train a new model entirely from scratch). > We use a pre-trained GP
128.
by zxexz 2y ago
Feeding the last layer back as the input embedding has been done many times, e.g. Transformer-XL. The models are trained like this, it's not like they're taking a pre-trained Llama and just feeding it to itself. It's a simple
129.
by zxexz 2y ago
The issue mailing list issue tracker works quite well.
130.
by zxexz 2y ago
I do this with the federal register + the comments and attachments. Probably the most informative govt. website.
131.
Mike's PBX Cookbook(pbxbook.com)
2 points by zxexz 2y ago|0 comments
132.
by zxexz 2y ago
Oh cool, they have a new location! I missed poking around after shows at the Somerville Theater.
133.
by zxexz 2y ago
Yep. That technique works very well. Surprised that it’s not more widely used.
134.
by zxexz 2y ago
The article is about a story involving elephants, and OP’s referenced zoo.
135.
by zxexz 2y ago
Did not know his was neuroendocrine. But still, something or somebody led him down that path (or rather, a multitude of someones and somethings, over who-knows-what timeframe). It's sad - not because it was Steve Jobs, but because it
136.
by zxexz 2y ago
Interesting, I think $100M is totally enough to train a SotA "foundation model". It's all in the use case. I'd love to hear explicit arguments against this.
137.
by zxexz 2y ago
The best time in history to have cancer is now, and we need to make sure that statement continues to be true for every year yet to come. Having cancer is still some of the worst news an individual can hear. And it’s nigh but inevitable; we
138.
by zxexz 2y ago
Is it in a position where, if remote work would be possible, regardless of corporate policy?
139.
by zxexz 2y ago
I’d believe either, though the latter feels tiny. Pandora is still better than any of the others listed for discovering new music, IMO. Spotify always seems too conservative with its suggestions, making it worse for discovering new music.
140.
by zxexz 2y ago
F# is exactly this, and a pleasure to work with.
141.
by zxexz 2y ago
Yes, I was. Thank you both for pointing this out.
142.
by zxexz 2y ago
I just spent dug in to this and, wow, I was wrong! Most all the negatives that occur with parenteral and oral routes appear to be absent in the intranasal form. And there is quite a lot of research to back that up. Thank you for correcting
143.
by zxexz 2y ago
As someone who really enjoys Haskell, I used to think like that. But I realized for problems like parsing, it really is just excellent.
144.
by zxexz 2y ago
Corticosteroids are powerful substances, and have lots of potential adverse effects - and long-term usage can wreak havoc. The physiological side-effects of corticosteroid withdrawal can be quite awful. They are amazing, necessary, drugs fo
145.
by zxexz 2y ago
Thanks for sharing! I'll give it a read tomorrow - I do not appear to have read this. I really do wish there were good places for randos like me to discuss this stuff casually. I'm in so many slack, discord, etc. channels but none
146.
by zxexz 2y ago
Sorry if that was ridiculously vague. I don't know a ton about the state of the art, and I'm really not sure there is one - the papers just seem to get more terminology-dense and the research mostly just seems to end up developi
147.
by zxexz 2y ago
Check out PyKEEN [0] and go wild. I like to train a bunch of random models and "overfit" them to the extreme (in my mind overfitting them is the point for this task, you want dense, compressed knowledge). Resize the input and ou
148.
by zxexz 2y ago
I can't imagine training took more than a day with 8 A100 even with that vocab size [0] (does lightning do implicit vocab extension maybe?) and a batch size of 1 [1] or 64 [2] or 4096 [3] (I have not trawled through the repo and other
149.
by zxexz 2y ago
I've seen some very impressive results just embedding a pre-trained KGE model into a transformer model, and letting it "learn" to query it (I've just used heterogenous loss functions during training with "classifier
150.
by zxexz 2y ago
I just saw your response to this - that's an excellent and concise way to explain this. Thank you. There're many categories of violence, and semantic violence can evoke any and all of those.

More ›