4 ms·

> The larger context window (200k tokens vs ~16k) Just to add some clarification - the newer GPT4 models from OpenAI have 128k context windows[1]. I regularly

by biddit 3y ago

> The larger context window (200k tokens vs ~16k) Just to add some clarification - the newer GPT4 models from OpenAI have 128k context windows[1]. I regularly load in the entirety of my React/Django project, via Aider. 1. https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turb...

danielbln 3y ago
GPT-4 has much worse recall compared to Claude-3 though, compare these two haystack tests: https://github.com/gkamradt/LLMTest_NeedleInAHaystack/raw/main/img/GPT_4_testing.png https://github.com/gkamradt/LLMTest_NeedleInAHaystack/raw/ma... https://cdn.arstechnica.net/wp-content/uploads/2024/03/claude3_chart6.-640x317.png https://cdn.arstechnica.net/wp-content/uploads/2024/03/claud...
- aubanel 3y ago
  Be aware the haystack test is not good at all (in its current form). It's a single piece of information inserted in the same text each time, a very poor measurement of how well the LLM can retrieve info.
  - bufferoverflow 3y ago
    Seems like a very good test for recall.
    - aubanel 3y ago
      Even in the most restrictive définition of recall as in "retrieve a short contiguous piece of information inside an unrelated context", it's not that good. It's always the exact same needle inserted in the exact same context. Not the slightest variation apart from the location of the needle. Then if you want to test for recall of sparse information or multi-hop information, it's useless.
      - BA2255 2y ago
        For my education, how do you use the 200k contenxt the normal chats like Poe, or chatgpt don't accept longer than 4k maximum. Do you use them in specific Playgrounds or other places?
        aubanel 2y ago
        The calls with long context are done through specific APIs, that you can call for instance in Python or Javascript. Here's a quick start guide with OpenAI: https://platform.openai.com/docs/quickstart?context=python https://platform.openai.com/docs/quickstart?context=python

by biddit 3y ago

danielbln 3y ago

GPT-4 has much worse recall compared to Claude-3 though, compare these two haystack tests: https://github.com/gkamradt/LLMTest_NeedleInAHaystack/raw/main/img/GPT_4_testing.png https://github.com/gkamradt/LLMTest_NeedleInAHaystack/raw/ma... https://cdn.arstechnica.net/wp-content/uploads/2024/03/claude3_chart6.-640x317.png https://cdn.arstechnica.net/wp-content/uploads/2024/03/claud...

aubanel 3y ago
Be aware the haystack test is not good at all (in its current form). It's a single piece of information inserted in the same text each time, a very poor measurement of how well the LLM can retrieve info.
- bufferoverflow 3y ago
  Seems like a very good test for recall.
  - aubanel 3y ago
    Even in the most restrictive définition of recall as in "retrieve a short contiguous piece of information inside an unrelated context", it's not that good. It's always the exact same needle inserted in the exact same context. Not the slightest variation apart from the location of the needle. Then if you want to test for recall of sparse information or multi-hop information, it's useless.
    - BA2255 2y ago
      For my education, how do you use the 200k contenxt the normal chats like Poe, or chatgpt don't accept longer than 4k maximum. Do you use them in specific Playgrounds or other places?
      - aubanel 2y ago
        The calls with long context are done through specific APIs, that you can call for instance in Python or Javascript. Here's a quick start guide with OpenAI: https://platform.openai.com/docs/quickstart?context=python https://platform.openai.com/docs/quickstart?context=python