MichaelRazum

12 ms·

31.
by MichaelRazum 2y ago
Kind of have the same intuition. However, to play devil's advocate, the human data and knowledge far exceeds what a single human can know and learn - so it would make it already superhuman.
32.
by MichaelRazum 2y ago
is it not similar to what happens in the US? https://m.youtube.com/watch?v=cTokizGa6zQ
33.
by MichaelRazum 3y ago
Thanks a lot for the hint :)! It awesome that it might run even on a MacBook, actually this is a reason to switch to Mac. Seems, there is nothing similar for a PC laptop with linux or windows.
34.
by MichaelRazum 3y ago
If I may ask, how do you load such big models? 300gb seems like a lot to play around with.
35.
by MichaelRazum 3y ago
This is awesome to bootstrap some ideas. The question is can it work with (large) existing code bases or modify it's own code. Guess a good test would be, can it reproduce Devin;)
36.
by MichaelRazum 3y ago
The Problem is math=!math. To be honest, this might be very challenging. BUT it may be possible. Just try entry level Analysis or Linear algebra. They often start kind of from scratch. How to construct numbers and so on. You could go very s
37.
by MichaelRazum 3y ago
Exactly, same thought. Actually I would expect, that they trained each expert separately and later together, since you need to train the router network as well. I'm far from an expert in LLMs. But this would be interesting to know, esp
38.
by MichaelRazum 3y ago
Maybe the guy is overdoing it (obviously). On the other hand, it seems that he put a lot of research into optimal nutrition. So would consider the meals pretty solid.
39.
by MichaelRazum 3y ago
Human Rights Watch is biased https://en.wikipedia.org/wiki/Criticism_of_Human_Rights_Watc...
40.
by MichaelRazum 3y ago
Maybe the right title would be a (statistical) physics perspective...
41.
by MichaelRazum 3y ago
Exactly, he could go and train grog. But seriously, some AI companies raise with ease 100MM or even more. Guess Ilya would have no trouble to raise the same amount or even more. He knows for sure what is needed to get to GPT4 level.
42.
by MichaelRazum 3y ago
It costs about 20k$ as far as I understand. Why is it so expensive?
43.
by MichaelRazum 3y ago
Maybe, if you already reached the atmosphere with it. Obviously, current AI tech scales with compute. It "may" be the case, that just more compute is enough to reach the moon..
44.
by MichaelRazum 3y ago
As I understand a H100 (SXM) gets you about 4 Petaflops on FP8. So 150k H100 is about 600 Exaflops on FP8. That is huge. Maybe enough to get AGI?
45.
by MichaelRazum 3y ago
Same thought. For 20k$ you can get a car. What you get here, at least looks like, a roomba with an arm gripper. I understand why it is so expensive, since it is not a mass product. Or in other words, it should be 10x cheaper, if it would be
46.
by MichaelRazum 3y ago
Just curious. What type of "stuff" is discussed in a board? I mean, if it is about methods/approaches/techniques that worked well, then msft gets the full insight, or do I miss something?
47.
by MichaelRazum 3y ago
Have seen an interview with someone from openai. At least they claimed, that they are far away from running out of data and they are able to generate data if needed.
48.
by MichaelRazum 3y ago
Was thinking the same, so 1T might bring you to the league of GPT-4. Actually in the best case, since it seems that meta, google, openai and so on have the most talent. Anyway, to bring it to the next level, how big should it be? Maybe 10T
49.
by MichaelRazum 3y ago
You could say the same about any person on the top. In general CEO's do not do research. Still they are critical for success. By the way the AI scientists get a lot of respect and admiration see Ilya for example.
50.
by MichaelRazum 3y ago
Haha, agree, it would take longer for sure. What I meant is, most likely assuming that you are using pytorch / jax you could code down the model pretty fast. Just compare it to llama, sure it is far behind, but the llama model is under
51.
by MichaelRazum 3y ago
I think you overestimate the technical part. Just speculating (no inside, no expert), but I would assume that the models are pretty "easy" and can be coded in few days. There are for sure some tweaks to the standard transformer ar
52.
by MichaelRazum 3y ago
Never change a good running system. OpenAI ran quite good, even if things heated up, it sound kind of stupid to fire the CEO when the company is winning on every front.
53.
by MichaelRazum 3y ago
Guess you are right. Anyway in my opinion, it actually doesn't matter where exactly the ads appear. I think to be fair, you would have to compare the feeds.
54.
by MichaelRazum 3y ago
As a user of Twitter/X, Instagram, and to some extent TikTok, I can say that, at least for me, Twitter/X is much cleaner and more aligned with the people I follow. In contrast, Instagram Reels are annoying; even after reporting ce
55.
by MichaelRazum 3y ago
which?
56.
by MichaelRazum 3y ago
I took a quick look into the paper. To be honest (I'm far from being an expert). As I understand it seems they took some toy models, and tried to show that the models do not generalize out of sample. To me it is unclear what does it sa
57.
by MichaelRazum 3y ago
Exactly, if something come out - guess it would be a funny nightmare to the physic society. As you said, it doesn't seem to be the case a right now. On the personal level, maybe you are right - but it doesn't matter, if he is righ
58.
by MichaelRazum 3y ago
Guess two reasons - the theory is not mainstream, guess it is not attractive enough to study it right now - the theory is not able to make any new predictions (yet). This has to change I think to get traction.
59.
by MichaelRazum 3y ago
True. Without exercises I would say you learned close to nothing.
60.
by MichaelRazum 3y ago
Exactly just luck and money. For sure he is just dumb. Just a dumb lucky guy who somehow got into a phd program for physics in some dumb university...

More ›