p1esk

40 ms·

841.
by p1esk 3y ago
I said “scale is all you need for transformers ”. That has been true since GPT1. The best way to improve our best model today still seems to be “make it larger and train it on more data”. If you disagree please suggest a better way, or at
842.
by p1esk 3y ago
Scale is all we need for transformers (so far). It might also be all we need for ape brains. It’s not all we need for whatever elephant or dolphin brains evolved from. When this stops being the case for transformers, we will need something
843.
by p1esk 3y ago
Ok, so how do humans solve a problem? Isn’t it also a sequential, step by step process, even if not expressed explicitly in words? What if instead of words a model would show you images to solve a problem? Would it change anything?
844.
by p1esk 3y ago
So what happens if I don’t have internet access? Windows refuses to install?
845.
by p1esk 3y ago
Other animals with larger brains might have other bottlenecks preventing them from reaching full potential of their intelligence. Neanderthals might have been smarter than us, but went extinct for reasons not related to intelligence. But my
846.
by p1esk 3y ago
I think when I write, so the machine must also think when it writes. What is it exactly you do when you “think”? And how is it different from what LLM does? Not saying it’s not different, just asking.
847.
by p1esk 3y ago
I prefer Hanlon’s razor.
848.
by p1esk 3y ago
What’s the main difference between an ape’s brain and a human brain? Scale. So that’s the train we’re riding at the moment. No roadblocks yet, aside from cost.
849.
by p1esk 3y ago
Hallucinations are an interesting problem - in both humans and statistical models. If we asked an average person 500 years ago how the universe works, they would have confidently told you the earth is flat and it rests on a giant turtle (or
850.
by p1esk 3y ago
I agree with your points but they are orthogonal to mine. A smart person might be more likely to say “I don’t know” than a stupid person. Also, a smart bullshit artist in your example does not hallucinate - he knows what he’s doing.
851.
by p1esk 3y ago
This doesn’t make sense - there are literally thousands of academic AI research labs who are severely limited by compute resources. If anything could work better than transformers and require less compute they would be all over that.
852.
by p1esk 3y ago
No technique can get you the news other than actually searching for and then parsing the published news.
853.
by p1esk 3y ago
Would it be nice if you no longer know who you are (as a person), or what you are capable of? Until some kind of personality engineering methods emerge to shape your development you might not be in control of the changes.
854.
by p1esk 3y ago
The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.
855.
by p1esk 3y ago
I'm guessing the vast majority of people with dementia cannot afford paying $11k/mo. What are their options? Is there an option of paying (much) less to receive a fraction of 24/7 care?
856.
by p1esk 3y ago
I am happy with who I am today.
857.
by p1esk 3y ago
No, it’s kind of like living when you are 4 years old. Very different than when you’re 40.
858.
by p1esk 3y ago
Long lasting neuronal plasticity sounds a bit scary. It might mean your personality could gradually change.
859.
by p1esk 3y ago
Do you really need 24 hour coverage? Are these patients really such a danger to themselves or others that you have to keep an eye on them at all times day and night? It seems to me that even 4 hour a day care would already result in a huge
860.
by p1esk 3y ago
Somehow I think if the standard salary for this kind of work is 50k/year (~4k/mo), it shouldn’t be too hard to find people willing to do it for double that (8k/mo) especially if it means living rent free.
861.
by p1esk 3y ago
We are talking about increasing the intercard bandwidth, assuming that’s a bottleneck. It can be done by either increasing pcie bandwidth, or using nvlink. If you use nvlink, increasing pcie does not provide any additional benefit because n
862.
by p1esk 3y ago
Why do you need x16 pcie slots if you can use nvlink?
863.
by p1esk 3y ago
I might not care about what my friends do for a living (I do though), but I definitely care a lot about what I do for a living.
864.
by p1esk 3y ago
I spoke to someone recently who believes poor people will be gradually killed off as the elites who control the robots won't have any use for them (the people). I don't share such an extreme view (yet), but I can't quite rule
865.
by p1esk 3y ago
Why? I see your book was mentioned in the article, but I don't see what's strange about it.
866.
by p1esk 3y ago
Jim Keller works for Tenstorrent - direct Nvidia competitor.
867.
by p1esk 3y ago
All these tasks you listed - what makes you think an average CEO knows how to do any of them? How do you know if the company is doing well because of the CEO or despite them?
868.
by p1esk 3y ago
Has anyone tried laying off most of the executive team - instead of the workers?
869.
by p1esk 3y ago
H200 has 141GB, B100 (out next month) will probably have even more. How much memory do you need?
870.
by p1esk 3y ago
https://twitter.com/yar_vol/status/1760314018575634842

More ›