4 ms·

And how much data can you give it? I'm not up to date with these things because I haven't found them useful. But with what you said, and previous limitations i

by tjoff 2y ago

And how much data can you give it? I'm not up to date with these things because I haven't found them useful. But with what you said, and previous limitations in how much data they can retain essentially makes them pretty darn useless for that task. Great learning tool on common subjects you don't know, such as learning a new programming-language. Also great for inspiration etc. But that's pretty much it? Don't get me wrong, that is mindblowingly impressive but at the same time, for the tasks in front of me it has just been a distracting toy wasting my time.

ben_w 2y ago
> And how much data can you give it? 128,000 tokens, which is about the same as a decent sized book. Their other models can also be fine-tuned, which is kinda unbounded but also has scaling issues so presumably "a significant percentage of the training set" before diminishing returns.
pimeys 2y ago
It is great for proof-reading text if you are not a native English speaker. Things like removing passive voice. Just give it your text and you get a corrected version out. Use a cli tool to automate this from the cli. Ollama for local models, llm for openai.
MPSimmons 2y ago
>And how much data can you give it? Well, theoretically you can give it up to the context size minus 4k tokens, because the maximum it can output is 4k. In practice, though, its ability to effectively recall information in the prompt drops off. Some people have studied this a bit - here's one such person: https://gritdaily.com/impact-prompt-length-llm-performance/ https://gritdaily.com/impact-prompt-length-llm-performance/
- jaredsohn 2y ago
  You should be able to provide more data than that in the input if the output doesn't use the full 4k tokens. So limit is context_size minus expected length of output.
Workaccount2 2y ago
People never talk about Gemini, and frankly it's output is often the worst of SOTA models, but it's 2M context window is insane. You can drop a few textbooks into the context window before you start asking questions. This dramatically improves output quality, however inference does take much much longer at large context lengths.

by tjoff 2y ago

ben_w 2y ago

> And how much data can you give it? 128,000 tokens, which is about the same as a decent sized book. Their other models can also be fine-tuned, which is kinda unbounded but also has scaling issues so presumably "a significant percentage of the training set" before diminishing returns.

pimeys 2y ago

It is great for proof-reading text if you are not a native English speaker. Things like removing passive voice. Just give it your text and you get a corrected version out. Use a cli tool to automate this from the cli. Ollama for local models, llm for openai.

MPSimmons 2y ago

>And how much data can you give it? Well, theoretically you can give it up to the context size minus 4k tokens, because the maximum it can output is 4k. In practice, though, its ability to effectively recall information in the prompt drops off. Some people have studied this a bit - here's one such person: https://gritdaily.com/impact-prompt-length-llm-performance/ https://gritdaily.com/impact-prompt-length-llm-performance/

jaredsohn 2y ago
You should be able to provide more data than that in the input if the output doesn't use the full 4k tokens. So limit is context_size minus expected length of output.

Workaccount2 2y ago

People never talk about Gemini, and frankly it's output is often the worst of SOTA models, but it's 2M context window is insane. You can drop a few textbooks into the context window before you start asking questions. This dramatically improves output quality, however inference does take much much longer at large context lengths.