3 ms·

I mean I just read that article and it doesn't seem like a lot will change, sure it can summarize a whole book, or read a larger chunk of code to do thing with,

I mean I just read that article and it doesn't seem like a lot will change, sure it can summarize a whole book, or read a larger chunk of code to do thing with, but didn't really see it talk about taking things to "the next level" so to speak. The researchers are also, excited. We’re especially motivated by applications that could benefit from longer-sequence models – high-resolution imaging, new modalities of data, language models that can read entire books. Imagine giving a language model an entire book and having it summarize the plot, or conditioning a code generation model on all the code you’ve ever written. The possibilities are wild – and we’re excited.

Herval_freire 4y ago
But this research came out mere weeks after the release of gpt-4. That is in itself rapid. If small incremental changes like this continue on a sort of monthly basis, the trendline points towards something that's not a dead end. That's my view of it. As with most technology there isn't necessarily always a constant influx of inflection points and paradigm shifts. Improvement will likely creep up on us incrementally. Suddenly one day its clearly more intelligent then a human and we can't point to when it happened.
- deleted 4y ago
  [deleted]
- jamilton 4y ago
  GPT-4's release doesn't seem like the relevant time marker, since nothing in the article builds on it or depends on it. The paper for H3 was submitted in December 2022. The pace the last few years definitely seems rapid, just don't want there to be a false impression.