thatguysaguy

9 ms·

91.
by thatguysaguy 3y ago
I think the critical thing is you need some ground truth way of evaluating the synthetic data. You can generate 100 programs with your LLM and filter to the 1-2 that solve the problem, but there's not an equivalent option for things li
92.
by thatguysaguy 3y ago
Another time he's talking and typing at the same time, but says a different thing than he "types".
93.
by thatguysaguy 3y ago
Def cloud, we don't have the tech to run what they're doing on that small of a device atm. (At least not at that price point).
94.
by thatguysaguy 3y ago
I think an acquisition does seem pretty likely actually. Google still has yet to make assistant do anything interesting, despite leading the way on the research side of things. Also keep in mind that they lost a ton of AI talent to startups
95.
by thatguysaguy 3y ago
I have no idea whether his stuff will work but I'm glad people are trying wacky stuff to find out.
96.
by thatguysaguy 3y ago
The term "AI Agent" is currently being used a lot to describe the setup where you have an LLM doing multiple generation rounds using tools etc so it can interact with an environment or other LLMs or whatever. Feel free to pass you
97.
by thatguysaguy 3y ago
I guess I can understand that POV even if I don't hold it myself.
98.
by thatguysaguy 3y ago
It's so strange to me that the first example is one where the devs went out of their way to help the users. There's no world in which that's maliciousness or anything of the sort.
99.
by thatguysaguy 3y ago
I'm kinda unclear on why I wouldn't want these to be able to access my site, what's the reasoning?
100.
by thatguysaguy 3y ago
This is a bit confusing since "AI Agent" has come to mean a specific thing, whereas this is about values of the User-Agent field.
101.
by thatguysaguy 3y ago
This only lists first moments, but Adam stores estimates of first and second moments.
102.
by thatguysaguy 3y ago
That's interesting because I remember testing LSTMs for language modeling on some dataset (probably PTB), and finding that they got lower perplexity left-to-right than right-to-left.
103.
by thatguysaguy 3y ago
I think this misses important history. This was a machine translation paper, and we were already using seq2seq RNNs with attention at the time. They didn't coin the term attention, they just realized that you could use attention from a
104.
by thatguysaguy 3y ago
Self-attention and positional embeddings are ancient at this point (i.e., more than 5 years old). Here are a couple examples of methods: https://arxiv.org/abs/2112.06749 https://arxiv.org/abs/2205.
105.
by thatguysaguy 3y ago
People have been trying to get non-autoregressive generation working for years. There are some methods which work okay but they're always behind AR generation. And yeah they do look very diffusion-esque. One of the issues is that you&#
106.
by thatguysaguy 3y ago
DPO should be listed as well: https://arxiv.org/abs/2305.18290 It's extremely zeitgeisty atm
107.
Ask HN: Advice on how to get a consulting pipeline for ML/NLP started
2 points by thatguysaguy 3y ago|2 comments
108.
by thatguysaguy 3y ago
Well, my understanding is that some theories of physics predict the existence of monopoles, and some say they can't exist. Finding a real monopole would have an impact on our understanding of physical reality, whereas this is more like
109.
by thatguysaguy 3y ago
It was super weird to see him align with the "regulate AI people" after years of being one of the main "this is going nowhere" guys
110.
by thatguysaguy 3y ago
They all say he's good at what he does, but none of them actually sound like he likes the guy.
111.
by thatguysaguy 3y ago
Are quant firms positive sum for society? I can imagine that some trading leads to goods being priced more efficiently or w/e but I doubt the level of alpha these firms are chasing has positive externalities. If not, you should shouldn
112.
by thatguysaguy 3y ago
They said single TPU machine to be fair, which means like 8 TPUs (still impressive)
113.
by thatguysaguy 3y ago
Memory bandwidth/latency, especially when you're at smaller batch sizes.
114.
by thatguysaguy 3y ago
No one else is really capable of making this technology at the level of the US, besides maybe China. Europe has the brainpower for sure, but they don't seem to be able to leverage it into actual innovation.
115.
by thatguysaguy 3y ago
You can see him talk to an actual physicist here: https://www.youtube.com/watch?v=0bMYtEKjHs0 Sean Carroll is more of a "just let the guest say what they want to" interviewer, so he doesn't grill him very har
116.
by thatguysaguy 3y ago
The idea is that it penalizes speculators for just holding onto property without putting it to work. I am not an economist so I don't know which of the pressures would win out.
117.
by thatguysaguy 3y ago
You're right, but: I think we could imagine finding something like "if the universe is infinite we can describe it with this simple set of laws, otherwise we need to use this much more complex set of laws." That would at leas
118.
by thatguysaguy 3y ago
Yeah loops don't get their own scopes (unless you add one using this thing I made as a joke: https://github.com/davisyoshida/360blockscope )
119.
by thatguysaguy 3y ago
Yeah I mean this argument seems to preclude anyone but me being conscious (bad news for you other commenters).