5 ms·

There's probably a reversal of that trend when you know a lot about it. I don't know how you could go through building your own GPT-2 and not see it as an incre

There's probably a reversal of that trend when you know a lot about it. I don't know how you could go through building your own GPT-2 and not see it as an incredible technology and useful advancement that is going to give us an incredible number of insights and tools. I still can't get over how soon people just accepted that you can universally translate all languages, that audio can be accurately transcribed, that you can speak a sentence and get pretty good research on a topic. These things are all actually insane and everyone has just decided to ignore this and focus on the aesthetic grossness of some of the things people are trying to get them to do.

aprilthird2021 2y ago
I mean, if you are willing to accept mistakes, then a lot of this was doable a while ago. Google Translate has been around a long time (and it's improved due to transformers but it was pretty good a long time ago too). Wikipedia has been around forever. Siri and other such agents have been doing audio transcription well long before. So all of these have improved. But they aren't completely novel
- Izkata 2y ago
  There were shareware Windows programs for audio transcription and even controlling the computer with your voice back in the 90s, I had some fun playing with one of them around 1999/2000. You had to train it to your voice by reading some text it gave you, but after that was pretty good. The modern stuff is definitely better untrained, but I'm not so sure about after training, though it has been a long time since I've used it.
  - Drew_ 2y ago
    Yeah I remember discovering the voice controls built into Windows Vista when I was a kid and being blown away by it.
vrighter 2y ago
Because the only time I can be confident in the answers it gives me, is when I already knew the answer before asking, which makes using the LLM pointless. Anything else would require me to do research to fact check it anyway. Might as well skip the LLM and do the research directly. This is what a lot of people forget about LLMs, if you didn't already know what the answer should be, then you won't know when it's giving you crappy responses. And if you do know what you are talking about and actually do catch it giving wrong information (or think you did), it'll just say "whoops.... you're right!". Sometimes, even if you aren't. The only things I can feel confident asking an LLM is stuff that I already know.

aprilthird2021 2y ago

I mean, if you are willing to accept mistakes, then a lot of this was doable a while ago. Google Translate has been around a long time (and it's improved due to transformers but it was pretty good a long time ago too). Wikipedia has been around forever. Siri and other such agents have been doing audio transcription well long before. So all of these have improved. But they aren't completely novel

Izkata 2y ago
There were shareware Windows programs for audio transcription and even controlling the computer with your voice back in the 90s, I had some fun playing with one of them around 1999/2000. You had to train it to your voice by reading some text it gave you, but after that was pretty good. The modern stuff is definitely better untrained, but I'm not so sure about after training, though it has been a long time since I've used it.
- Drew_ 2y ago
  Yeah I remember discovering the voice controls built into Windows Vista when I was a kid and being blown away by it.

vrighter 2y ago

Because the only time I can be confident in the answers it gives me, is when I already knew the answer before asking, which makes using the LLM pointless. Anything else would require me to do research to fact check it anyway. Might as well skip the LLM and do the research directly. This is what a lot of people forget about LLMs, if you didn't already know what the answer should be, then you won't know when it's giving you crappy responses. And if you do know what you are talking about and actually do catch it giving wrong information (or think you did), it'll just say "whoops.... you're right!". Sometimes, even if you aren't. The only things I can feel confident asking an LLM is stuff that I already know.