zwaps

32 ms·

271.
by zwaps 4y ago
Am I stupid? Is there a paper for those models? What sizes? What architecture and setup? What kind of instruction tuning? RLHF? Where are the infos?
272.
by zwaps 4y ago
Embeddings can go beyond a contextual definition of meaning. You can use (transformer) LLMs to get a "word sense" aka its predicted probability distribution, which also induces a network of associations (not via context, but via w
273.
by zwaps 4y ago
You should check out this https://cats.for.ai/program/
274.
by zwaps 4y ago
My favorite is: I am training GPT to... what they are doing: adding a prompt
275.
by zwaps 4y ago
I get your point, but please do read the training logs for Meta's OPT, it's some Greek drama I tell ya
276.
by zwaps 4y ago
would be better if it'd work on Firefox tho
277.
by zwaps 4y ago
It’s a blog with photos? Even with your description I don’t get it. If I make a website, which is easy to do nowadays, I can also share that link. What exactly is the revolution here?
278.
by zwaps 4y ago
Lee, please, less blatant the ads. Thank ye
279.
by zwaps 4y ago
Had the same with todoist: account deleted, still pulled money. To be fair, I got it resolved + my money back after a handful of e-mails.
280.
by zwaps 4y ago
If you now add a "made in X" indicator, I guess I will order in Switzerland from now on
281.
by zwaps 4y ago
I think search is hard generally. I switched to an M1 Mac pro recently, and it has its own set of challenges. For instance, there is no start menu but a search bar called Spotlight. You do your quick math in there (where you'd call cal
282.
by zwaps 4y ago
Mercedes Benz
283.
by zwaps 4y ago
Apple has a point tho: what PC laptop or what typical PC for that matter (and that price) can give you 96 GB of VRAM? If you could run ML models properly on that machine, it would be pretty nice for inference on larger models. Now, of cours
284.
by zwaps 4y ago
But what then? Python is the only language everyone can agree on. I'd love to code in Julia, and I'd happily use something like Rust. But realistically? Everyone does Python, and getting Python code out of critical path's is
285.
by zwaps 4y ago
Ah yes, memgraph. I argued with them here about another bullshit benchmark https://news.ycombinator.com/item?id=33717766 they did reply tho
286.
by zwaps 4y ago
This is what ChatGPT does, for instance.
287.
by zwaps 4y ago
Gpt3 is a graph neural network with added recursive information through positioninal encoding. It outputs sequentially, but I am not sure why a RNN would be required beyond that. I would agree that the manner of reasoning must differ, since
288.
by zwaps 4y ago
What do you mean by current methods though? ChatGPT is trained very differently method wise than GPT3 itself. Whereas GPT3 was strongly based on sequential likelihood in a large corpus, ChatGPT is trained (i would claim) on producing outpu
289.
by zwaps 4y ago
So yes, you’ll probably want to take a model trained on math problems to forgoe the step with code. But then, note that writing correct programs is indeed a high level solution to a full text math problem, is it not? Going from there to sol
290.
by zwaps 4y ago
If you prompt it correctly it seems to be able to solve most basic math problems you throw at it. Here's a PNAS claiming human-level performance: https://www.pnas.org/doi/full/10.1073/pnas.2123433119 The
291.
by zwaps 4y ago
ChatGPT is trained via reinforcement learning to give answers that - ultimately - human non-experts would judge to be plausible. Answers were not trained to be correct or accurate - hence we should not expect ChatGPT to do well there. Socio
292.
by zwaps 4y ago
The main point here is that Large Language Models are crushing through barriers that people like Gary Marcus have previously deemed impossible to surpass. Check past episodes by these guys on Machine Learning Street Talk podcast (the whole
293.
by zwaps 4y ago
Power management and sleep (like, closing the lid of the laptop and it waking up correctly every time) has basically never worked for me on Linux machines - of which I had several.
294.
by zwaps 4y ago
I will (and have) happily worked 80 hours per week, if you compensate me. Instead, my real wage has decreased this year. Of course, this means I substitute free time for work. My work time budget has been reduced. Since real wages have been
295.
by zwaps 4y ago
I asked a Star Trek question (somewhat obscure but time wise and corpus wise clearly in the training set) and the answer was incredibly wrong … but the output sounded very confident and coherent so kudos! Now all we need are the right answe
296.
by zwaps 4y ago
No app that would be banned from app stores is likely to be successful in the long term. Without revenue in it, and influencers on it, only few would jump through the insane hoops to sideload an app nowadays.
297.
by zwaps 4y ago
In context learning is probably best framed as priming the model towards already learned relations. The model was trained previously. Machine Learning Street Talk and Yannik Kilchers coverage of this years Neurips conference give some good
298.
by zwaps 4y ago
So now prompting GPT3 with some context counts as "training ChatGPT"? What?
299.
by zwaps 4y ago
Whatsapp was hugely popular before Meta bought it. In fact, it lost quite a bit of users when Facebook bought it, as many tried to move their networks to Signal, Threema etc So what was said is exactly right: nobody trusts Meta, rightly so
300.
by zwaps 4y ago
If you are struggling with self-worth, please read very carefully the posts of people who actually knew Chris Seaton. People value him by how much of himself and his time he was willing to give to others, not by some absolute measure of his

More ›