davesque

18 ms·

271.
by davesque 3y ago
Based on my skimming the paper, am I correct in understanding that they came up with an elaborate collection of prompts that embed the text generated by GPT-2 as well as a representation of GPT-2's internal state? Then, in effect, the
272.
by davesque 3y ago
Some people appreciate being able to choose whether or not to continue browsing. Infinite scroll is not objectively better.
273.
by davesque 3y ago
As an aside, I feel like the whole notion of Kolmogorov complexity seems to just be moving the goal posts. With the classic example of the Mandelbrot fractal, is the representation of the fractal as an iterated formula really any more compa
274.
by davesque 3y ago
Am I the only one who has not been able to get ChatGPT to produce what I consider to be passable writing? I've only succeeded in getting ChatGPT to output completely predictable, unoriginal, disengaging prose. For example, try getti
275.
by davesque 4y ago
I'm a bit confused why LeCun, Ng, or anyone else is acting like this is an actual thing that could happen.
276.
by davesque 4y ago
Is it though? I remember feeling much more positive about society roughly 20 years ago.
277.
by davesque 4y ago
I feel like the whole ChatGPT bubble has really highlighted what feel like some fundamental shortcomings in the worldview that is represented in tech. That is, there seems to be a winner take all dynamic baked into the tech world. Maybe t
278.
by davesque 4y ago
Is your fear of failure actually a proxy for a rational assessment of risk? Because if you don't have much to fall back on, fearing failure is pretty healthy.
279.
by davesque 4y ago
Sounds interesting, but I'm really asking more of a technical question here than a philosophical one. Your comment seems a bit more high level than what I'm going for.
280.
by davesque 4y ago
I'd be interested in hearing people's takes on the simplest mathematical reason that transformers are better than/different from fully connected layers. My take is: Q = W_Q X K = W_K X A = Q^T K = (X^T W_Q^T) (
281.
by davesque 4y ago
> I will maintain that this industry is morally bankrupt and nearly entirely devoid of empathy. These are not the people who should be in charge of our future. Since the tone of your characterization is so absolute, why doesn't it a
282.
by davesque 4y ago
Yeah, but then what do you do after you measure it? Nothing? No, you make decisions differently so as not to offend whoever is part of the criteria. For example, can we agree that we don't want an "author_is_flat_earther"
283.
by davesque 4y ago
> Americans (let alone people elsewhere in the world) are already struggling I agree. And I agree with your overall sentiment about the risks of pursuing AGI. I'm as cynical as anyone about the likelihood that the average person wil
284.
by davesque 4y ago
> Hardly anyone in the tech community is capable of truly empathizing with the vast majority of people This is a gross exaggeration.
285.
by davesque 4y ago
This is exactly what I came here to say. Yes, GPT-4 may happen to give a correct answer. But what happens when it's wrong? And how would you know it's wrong without a specific background in some topic?
286.
by davesque 4y ago
Is this really a big problem? LoRa is a telecom thing. LoRA is a machine learning thing. Yeah, they're adjacent industries but still seems different enough to make it pretty easy to distinguish. I had never heard of the LoRa allian
287.
by davesque 4y ago
Time to get spiritual. For some reason, your comment made me think of an interesting conversation I had while I was on a walk once with a friend of mine who is a Chan Buddhist monk. We were walking by a hotel in Boulder, CO called the St. J
288.
by davesque 4y ago
Nice, so you demeaned the homeless in an attempt to demean people who had just lost their jobs. Good for you.
289.
by davesque 4y ago
The folks at OpenAI have already started doing this. Just the other day, their chief scientist was making noises along those lines.
290.
by davesque 4y ago
I'm with you on this. And I am absolutely the furthest possible thing from a climate skeptic. If anything, it's just a recognition that this kind of communication from official sources is clearly ineffective. The climate crisis is
291.
by davesque 4y ago
Honestly, DHH and Basecamp have both struck me as being kind of old news for a while now. And he's the only other person I've heard making noises about the cloud like it's a bad thing.
292.
by davesque 4y ago
Paywalled so couldn't read. Does this article mention who is doing this or give any numbers? I'm at a large, top 10 software vendor and this is not what I'm seeing at all.
293.
by davesque 4y ago
All I have to say is...wow.
294.
by davesque 4y ago
Wait, they're trying to say that this was all the fault of the depositors? No, sorry. While I often find it hard to empathize with wealthy tech bros, I don't have much trouble at all imagining myself doing the same thing.
295.
by davesque 4y ago
How does someone as smart as Sutskever come to believe we can possibly keep a lid on this knowledge? I can't imagine GPT-4 is doing anything much different than what has already been described in publicly available research. Also, does
296.
by davesque 4y ago
> We’ve had photographs for a long time, and people still do representational art of real places and things; we have superior industrial mass production and people do inferior handicrafts, etc.; we’ve tested your theory extensively and i
297.
by davesque 4y ago
> this shouldn't be hard to measure > see if they do better or worse over the next few years This doesn't strike me as entirely ethical. Also, it may be hard to gauge the long term effects of subtle misinformation generated
298.
by davesque 4y ago
We still lack insight into the nature of GPT's hallucinations. Without the supervision of an expert human, we don't know when these hallucinations are happening. Given that, is this really the right tool to "preserve"
299.
by davesque 4y ago
Yes, but human teachers can eventually come to understand why they made a mistake. And they probably are more able to communicate how confident they are in their answers. I think that's because human teachers have a mental model that t
300.
by davesque 4y ago
GPT-4 definitely seems to be doing better on a lot of benchmarks and that's impressive. But it still hallucinates facts and I don't think anyone really has a good understanding of when and how that happens. Given that, is it reall

More ›