itkovian_

7 ms·

31.
by itkovian_ 1y ago
Completely agree and think it’s a great summary. To summarize very succinctly; you’re chasing a moving target where the target changes based on how you move. There’s no ground truth to zero in on in value-based RL. You minimise a difference
32.
by itkovian_ 1y ago
Saying we should tokenize different modalities the same would be analogous to saying that in order to be really smart, a human has to listen with its eyes. At some point there has to be SOME modality specific preprocessing. The thing is in
33.
by itkovian_ 1y ago
I don’t want to bash the guy since he’s still in his phd, but it’s written in such a confident tone for something that is so all over the place that I think it’s fair game. Like a lot of the symbolic/embodied people, the issue is they
34.
by itkovian_ 2y ago
And in this categorization auto regressive llms are contrastive due to the cross entropy loss.
35.
by itkovian_ 2y ago
The fundamental distinction is usually made to contrastive approaches (i.e. make correct more likely, make everything else we just compared unlikely). Ebms are "only what is correct is more likely and the default for everything is unli
36.
by itkovian_ 2y ago
>This is due to the fact that LLMs are basically just giant look up maps with interpolation. This is obviously not true at this point except for the most loose definition of interpolation. >don't rely on things like differentiabi
37.
by itkovian_ 2y ago
I mean theres lots one could say here. Probably the most straightforward is igor danshenko, who was the primary source for the steel dossier, stating that he never intended for the claims to be taken seriously.
38.
by itkovian_ 2y ago
It's an extraordinary claim. I think the reason I dismiss it as unlikely as when I look back at the steel dossier and muler investigation 1) if there was something, it's very likely they would have found it then 2) in hindsight bo
39.
by itkovian_ 2y ago
All competitive open models today share a common property; someone spent a large amount of money to train them and then released the model for free. I don't understand why the argument continues to be we will have a rich ecosystem of b
40.
by itkovian_ 2y ago
Story is good example of a why there's no "killer app" for llms. They are the killer app. There's no stack to this tech. People don't want to admit this because of the massive concentration of power that becomes cle
41.
by itkovian_ 2y ago
"That is a phrase that I coined in a 2022 essay called “Deep Learning is Hitting a Wall,” which was about why scaling wouldn’t get us to AGI. And when I coined it, everybody dismissed me and said, “No, we’re not reaching diminishing re
42.
by itkovian_ 2y ago
Completely anecdotal; the amount of people that do not consume caffeine seems overrepresented among very very smart people (i.e. math profs, researchers etc)
43.
by itkovian_ 2y ago
Incredible how low usage is among lawyers. Does anyone have any intuition on why?
44.
by itkovian_ 2y ago
Here's an example of the type of question it is acheiving 20% on; The set of natural transformations between two functors F,G ⁣:C→DF,G:C→D can be expressed as the end Nat(F,G)≅∫AHomD(F(A),G(A)). Nat(F,G)≅∫A HomD (F(A),G(A)). Define set
45.
Protocol Learning, Protocol Models, and the Great Convergence(pluralisresearch.com)
1 points by itkovian_ 2y ago|0 comments
46.
by itkovian_ 2y ago
I remember when openai first raised and had the 100x cap and everyone said that was ridiculous and insane and of course they're not going to 100x from 1b... That would require them to become a 100b company!
47.
by itkovian_ 2y ago
Tsmc is mostly governed by Taiwanese who would like to maintain Taiwanese sovereignty
48.
by itkovian_ 2y ago
Tsmc will never allow the Arizona plant to be a viable replacement. They are extremely incentived to prevent this happening.
49.
by itkovian_ 2y ago
This is not true
50.
by itkovian_ 2y ago
Phsycohistory
51.
by itkovian_ 2y ago
I think gpt4o is probably doing some ocr as preprocessing. It's not really controversial to say the vmls today don't pick up fine grained details - we all know this. Can just look at the output of a vae to know this is true.
52.
by itkovian_ 2y ago
That's not what a kb is
53.
by itkovian_ 2y ago
Knowledge graphs where created to solve the problem of making natural,free flowing text machine processable. We now have a technology that completely understands natural free flowing text and can extract meaning. Why would going back to str
54.
Decentralized Training Looms(pluralisresearch.substack.com)
1 points by itkovian_ 2y ago|0 comments
55.
by itkovian_ 2y ago
Everything is if scaling keeps making the models better. If it does you don't train 50 gpt4s, you have the best model.
56.
by itkovian_ 2y ago
I just don't understand the implied argument here at all. We can do something that results in productivity gains, which people want or they wouldn't pay for it, but we don't want private companies to do this because we have d
57.
by itkovian_ 2y ago
This person has significantly less ml exp. than I do. I guess it's fine for me to totally dismiss their argument in the same way they dismiss anyone who has less than them.
58.
by itkovian_ 2y ago
Sometimes things really are as significant as they seem.
59.
by itkovian_ 2y ago
I think there's a reasonable argument that cohere might not capture the value and it might all go to the big players. But I can't understand how anyone still has the view llms themselves are not valuable.
60.
by itkovian_ 2y ago
One datpoint; I visited Nepal in 2016 and was overwhelmed by the general baseline happiness of everyone there. Everyone smiled. Then contrast returning to the West was stark. I went back in 2023 and while it's not gone it's waned.

More ›