4 ms·

I didn't really trust Google's 1m+ context. But I trust this paper and 1m+ context would be available everywhere. I know people complain about hardware and com

I didn't really trust Google's 1m+ context. But I trust this paper and 1m+ context would be available everywhere. I know people complain about hardware and compute resources. But, this is like complaining about Python resource usage in the early 90's. The development complexity & resources is far more expensive than chips on the long run. I am personally re-organizing my AI organization to move away from complex RAG setups and get comfortable with long-context workflows. Just to be clear - I also think that inference-optimized chips are the next frontier - Nvidia GPUs were designed & built in a different age than what’s going on now.

foobiekr 3y ago
This is 100% not true at all. Chips and electricity and the development resources would need to be enormous for your statement to be true. It's really not even true for your python example.
sebzim4500 3y ago
Quite a few people have access to Gemini's 1M context length and it does seem to work very well. E.g. can produce accurate scene-by-scene descriptions of hour long movies, can answer questions about very large codebases, etc.
- devinprater 3y ago
  This is a dream come true for me. Full movie descriptions! Or, uh, more likly for me, full video game playthrough descriptions!
  - TrueDuality 3y ago
    Legitimately curious, what is lacking from current playthrough and movie descriptions available now? For games the only things I feel like might not be represented well in text is descriptions of the observer/player reactions and maybe details about their specific choices through a game... Kind of like turning a choose your own adventure book into a normal book by tracing one path through it? I can see Q&A on a movie being useful but can't think of descriptions of the overall movie itself lacking... I'd definitely be worried about spoilers.
    - isaacfung 3y ago
      It may provide a way for AI to learn a complete new topic(a new programming language, stock investment, sport, games) in a zero shot way and immediately apply it by just feeding a youtube playlist/udemy course into an AI model.
    - devinprater 3y ago
      For a blind person, yeah some games, like Mario and Zelda, and older Final Fantasy games, are hard to understand without descriptions.
- antman 3y ago
  The problem is mostly not summarisation but needles in the haystack search with a combined answer
  - harshaxnim 3y ago
    Which Google claims Gemini is the best at so far
deleted 3y ago
[deleted]
rldjbpin 3y ago
This analogy disregards all the work that happens behind the scenes to make modern Python workload work. The current landscape is moving so fast that we don't stop and work on achieving the best version of a specific approach. One can argue that both RAG and having 1M+ context length is a symptom of a disconnect between what language models are capable of and what people think they can achieve with them. Within the next 6 months there might be yet another approach to provide LLMs custom memory, and we will be having the same discussion the way things are handled currently.

foobiekr 3y ago

This is 100% not true at all. Chips and electricity and the development resources would need to be enormous for your statement to be true. It's really not even true for your python example.

sebzim4500 3y ago

Quite a few people have access to Gemini's 1M context length and it does seem to work very well. E.g. can produce accurate scene-by-scene descriptions of hour long movies, can answer questions about very large codebases, etc.

devinprater 3y ago
This is a dream come true for me. Full movie descriptions! Or, uh, more likly for me, full video game playthrough descriptions!
- TrueDuality 3y ago
  Legitimately curious, what is lacking from current playthrough and movie descriptions available now? For games the only things I feel like might not be represented well in text is descriptions of the observer/player reactions and maybe details about their specific choices through a game... Kind of like turning a choose your own adventure book into a normal book by tracing one path through it? I can see Q&A on a movie being useful but can't think of descriptions of the overall movie itself lacking... I'd definitely be worried about spoilers.
  - isaacfung 3y ago
    It may provide a way for AI to learn a complete new topic(a new programming language, stock investment, sport, games) in a zero shot way and immediately apply it by just feeding a youtube playlist/udemy course into an AI model.
  - devinprater 3y ago
    For a blind person, yeah some games, like Mario and Zelda, and older Final Fantasy games, are hard to understand without descriptions.
antman 3y ago
The problem is mostly not summarisation but needles in the haystack search with a combined answer
- harshaxnim 3y ago
  Which Google claims Gemini is the best at so far

deleted 3y ago

[deleted]

rldjbpin 3y ago

This analogy disregards all the work that happens behind the scenes to make modern Python workload work. The current landscape is moving so fast that we don't stop and work on achieving the best version of a specific approach. One can argue that both RAG and having 1M+ context length is a symptom of a disconnect between what language models are capable of and what people think they can achieve with them. Within the next 6 months there might be yet another approach to provide LLMs custom memory, and we will be having the same discussion the way things are handled currently.