3 ms·

If I had no contact with society from the 29th of November 2022 (the day before ChatGPT was released according to Wikipedia) and came back today to see the Open

If I had no contact with society from the 29th of November 2022 (the day before ChatGPT was released according to Wikipedia) and came back today to see the OpenAI keynote I would have lost my mind. The progress and usefulness of these products is absolutely incredible.

qingcharles 3y ago
I was in prison when ChatGPT came out. All I knew of it was a headline that flashed past really fast on CNN and I called my buddy and said "What the hell is Chat OPT?" I'd just finished reading The Singularity is Near for the second time too...
- freedomben 3y ago
  The singularity is near is a great book. Hilarious that you read that for the second time and then got out and saw chat gpt! I love kurzweil but his estimates of timeline are often pretty over optimistic, so I'd be really wondering.
  - cubefox 3y ago
    On Metaculus the arrival for weakly general AI was predicted for 2045 two years ago. Now it's at 2026. https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/ https://www.metaculus.com/questions/3479/date-weakly-general...
    - qingcharles 3y ago
      I would concur with those dates now. When I read the book the first time about three years ago I thought "2045" is about right. When I saw DALL-E 2 I thought "2030". When I saw GPT4 I thought "2026".
byteflip 3y ago
I've been in contact with society and I'm still losing my mind.
ssnistfajen 3y ago
I remember opening Twitter that night and seeing a bunch of tech people I follow sharing screenshots of conversations with a little green icon. I thought "oh neat, yet another chatbot fad to try out for 5 minutes". I could not have been more wrong.
torginus 3y ago
I'm sorry, what breakthrough feature did we see here? - Code interpreter, function calling were already possible on any sufficiently advanced LLM that could follow instructions well enough to output tokens in a rigidly parseable format, which could then be fed into a parser, and its output fed back to the LLM. It was clunky to do with online APIs like ChatGPT, but still eminently possible. - Custom chatbots were easy to build before, and services to build them (like Poe.com) existed before. - Likewise outputting JSON just requires a good instruction following AI, that can output token probabilities, along with a schema validator that always picks a token that results in schema-conforming JSON - GPT4-128k seems to be revolutionary, but Claude-100k already existed, and considering LLM evaluation is quadratic wrt context size, they are probably using some tricks to extend the context, they are not 'full' tokens (I'd be happy to be proven wrong). While having a huge context is useful, for coding, a 8k context can be enough with some elbow grease (like filling the context with a 2-3 deep recursive 'Go To Definition' for a given symbol), so that the AI receives the right context. - Dall-E 3 seems to be the most revolutionary, but after playing with it, it has much improved compositional ability over SD, but it's still prone to breakdowns Overall I feel like todays announcements were polish and refinement over last year's bombshell breakthroughs.
- awestroke 3y ago
  You're completely ignoring image analysis.
  - torginus 3y ago
    sorry, missed that one, yeah, that's new as well, but to be fair, I missed it because nobody else was talking about it.
- reissbaker 3y ago
  The most exciting announcements for me were: * GPT-4-128k. Sure, Claude exists, but it's closer to GPT-3.5 than 4 IMO. TBD how well 128k context works given that classic attention scales quadratically (so they're presumably using something else) but given how good OpenAI's models tend to be I'm willing to give them the benefit of the doubt. * Pushed GPT-3.5 finetuning to 16k context (up from 4k when it was released this summer). IME 3.5 finetunes are very useful, very fast, very cheap replacements for specific specialized tasks over GPT-4, and easily outperform GPT-4 for the right kind of tasks. The 4k context limit was a bit of a bummer. * New tts that to my ears sounds nearly equivalent to Eleven Labs or Play.ht, at one-tenth to one-twentieth the price (with zero monthly commitment). The Eleven Labs Discord is a bit of a bloodbath right now, most of the general chat is just people saying they're switching. (The Play.ht Discord is pretty dead most of the time anyway, so not much new since this morning.) I will say though that it's a bummer that the OpenAI tts doesn't have input streaming, only output streaming, so latency will likely be worse and you'll have to figure out some way to do chunking yourself which is fairly annoying, but for any kind of personalized use case (e.g. a bot talking to customers, as opposed to using pre-recorded snippets) a 10-20x price improvement is worth the extra pain and may be the difference between "neat prototype" and "shippable to production." Plus, massive price drops for OpenAI's existing products across the board, along with a legal defense fund to protect OpenAI customers from getting sued for using OpenAI models. If you're building an "OpenAI wrapper startup," today was a very good day. If you're competing with OpenAI, though... Oof.
yen223 3y ago
Does it boggle anyone else's mind that ChatGPT was released less than a year ago? It definitely feels like it was around lot longer than that.
- gardenhedge 3y ago
  And there's people (most people) who aren't using it
  - jayGlow 3y ago
    what's really crazy to me is how fast it became mundane, this technology is still mind blowing to me but most people no longer seem impressed.
    - btbuildem 3y ago
      Even we who are so impressed with it and use it frequently / build on it -- we don't have a clue of the true potential. It'll be some pimply intern somewhere that'll blow the lid off things with some ultra-clever-yet-painfully-obvious use case.

qingcharles 3y ago

I was in prison when ChatGPT came out. All I knew of it was a headline that flashed past really fast on CNN and I called my buddy and said "What the hell is Chat OPT?" I'd just finished reading The Singularity is Near for the second time too...

freedomben 3y ago
The singularity is near is a great book. Hilarious that you read that for the second time and then got out and saw chat gpt! I love kurzweil but his estimates of timeline are often pretty over optimistic, so I'd be really wondering.
- cubefox 3y ago
  On Metaculus the arrival for weakly general AI was predicted for 2045 two years ago. Now it's at 2026. https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/ https://www.metaculus.com/questions/3479/date-weakly-general...
  - qingcharles 3y ago
    I would concur with those dates now. When I read the book the first time about three years ago I thought "2045" is about right. When I saw DALL-E 2 I thought "2030". When I saw GPT4 I thought "2026".

byteflip 3y ago

I've been in contact with society and I'm still losing my mind.

ssnistfajen 3y ago

I remember opening Twitter that night and seeing a bunch of tech people I follow sharing screenshots of conversations with a little green icon. I thought "oh neat, yet another chatbot fad to try out for 5 minutes". I could not have been more wrong.

torginus 3y ago

I'm sorry, what breakthrough feature did we see here? - Code interpreter, function calling were already possible on any sufficiently advanced LLM that could follow instructions well enough to output tokens in a rigidly parseable format, which could then be fed into a parser, and its output fed back to the LLM. It was clunky to do with online APIs like ChatGPT, but still eminently possible. - Custom chatbots were easy to build before, and services to build them (like Poe.com) existed before. - Likewise outputting JSON just requires a good instruction following AI, that can output token probabilities, along with a schema validator that always picks a token that results in schema-conforming JSON - GPT4-128k seems to be revolutionary, but Claude-100k already existed, and considering LLM evaluation is quadratic wrt context size, they are probably using some tricks to extend the context, they are not 'full' tokens (I'd be happy to be proven wrong). While having a huge context is useful, for coding, a 8k context can be enough with some elbow grease (like filling the context with a 2-3 deep recursive 'Go To Definition' for a given symbol), so that the AI receives the right context. - Dall-E 3 seems to be the most revolutionary, but after playing with it, it has much improved compositional ability over SD, but it's still prone to breakdowns Overall I feel like todays announcements were polish and refinement over last year's bombshell breakthroughs.

awestroke 3y ago
You're completely ignoring image analysis.
- torginus 3y ago
  sorry, missed that one, yeah, that's new as well, but to be fair, I missed it because nobody else was talking about it.
reissbaker 3y ago
The most exciting announcements for me were: * GPT-4-128k. Sure, Claude exists, but it's closer to GPT-3.5 than 4 IMO. TBD how well 128k context works given that classic attention scales quadratically (so they're presumably using something else) but given how good OpenAI's models tend to be I'm willing to give them the benefit of the doubt. * Pushed GPT-3.5 finetuning to 16k context (up from 4k when it was released this summer). IME 3.5 finetunes are very useful, very fast, very cheap replacements for specific specialized tasks over GPT-4, and easily outperform GPT-4 for the right kind of tasks. The 4k context limit was a bit of a bummer. * New tts that to my ears sounds nearly equivalent to Eleven Labs or Play.ht, at one-tenth to one-twentieth the price (with zero monthly commitment). The Eleven Labs Discord is a bit of a bloodbath right now, most of the general chat is just people saying they're switching. (The Play.ht Discord is pretty dead most of the time anyway, so not much new since this morning.) I will say though that it's a bummer that the OpenAI tts doesn't have input streaming, only output streaming, so latency will likely be worse and you'll have to figure out some way to do chunking yourself which is fairly annoying, but for any kind of personalized use case (e.g. a bot talking to customers, as opposed to using pre-recorded snippets) a 10-20x price improvement is worth the extra pain and may be the difference between "neat prototype" and "shippable to production." Plus, massive price drops for OpenAI's existing products across the board, along with a legal defense fund to protect OpenAI customers from getting sued for using OpenAI models. If you're building an "OpenAI wrapper startup," today was a very good day. If you're competing with OpenAI, though... Oof.

yen223 3y ago

Does it boggle anyone else's mind that ChatGPT was released less than a year ago? It definitely feels like it was around lot longer than that.

gardenhedge 3y ago
And there's people (most people) who aren't using it
- jayGlow 3y ago
  what's really crazy to me is how fast it became mundane, this technology is still mind blowing to me but most people no longer seem impressed.
  - btbuildem 3y ago
    Even we who are so impressed with it and use it frequently / build on it -- we don't have a clue of the true potential. It'll be some pimply intern somewhere that'll blow the lid off things with some ultra-clever-yet-painfully-obvious use case.