4 ms·

In my tests LLaMa2-13B is useable for information extraction tasks and LLaMA2-70B is almost as good as GPT-4 (for IE). These models are the real thing. We can f

In my tests LLaMa2-13B is useable for information extraction tasks and LLaMA2-70B is almost as good as GPT-4 (for IE). These models are the real thing. We can fine-tune LLaMAs, unlike OpenAI's models. Now we can have privacy, control and lower prices. We can introduce guidance, KV caching and other tricks to improve the models. The enthusiasm around it reminds me of JavaScript framework wars of 10 years ago - tons of people innovating and debating approaches, lots of projects popping up, so much energy!

asabla 3y ago
> The enthusiasm around it reminds me of Javascript wars 10 years ago... so much energy! I kind of have the same feeling as well. With all this energy it's really hard to keep up with all new ideas, implementations, frameworks and services. Really excited for what this will bring us the next coming years
- sva_ 3y ago
  > With all this energy it's really hard to keep up with all new ideas, implementations, frameworks and services. The majority of them are mostly irrelevant. You just need to figure out which.
worldsavior 3y ago
It looks like it will always be a war like Android VS iOS only now it's with AI models.
pavlov 3y ago
> “The enthusiasm around it reminds me of JavaScript framework wars of 10 years ago” Hmm. If LLMs turned out like JS frameworks, that would mean that in ten years people will be saying: “Maybe we don’t really need all this expensive ceremony, honestly this could be done with vanilla if/else heuristics…?”
- spacebanana7 3y ago
  I can imagine a bloated world where 500B param models are used for tasks where 7B param modes perform adequately. At that time, there could be complaints on hacker news about messaging apps with autocomplete models that take up gigabytes.
  - pavlov 3y ago
    I could see a world where people turn to an LLM for a task where the hand-rolled solution could be a simple state machine or some nested switch statements. The irony would be that the LLM could write you that code, but if you don’t know to ask…
    - falcor84 3y ago
      The real irony would be if the AI uses an LLM to write the code the second time it sees the same request repeated, deploys it to effortlessly deal with all future requests, and goes back to playing Quake while collecting the full paycheck.
  - dmd 3y ago
    "How can I sum this column of numbers?" "IDK, throw it at the LLM"
    - azeirah 3y ago
      I do stuff like this sometimes when I have some csv or something and need it in JSON. Could easily be done with 1 line of bash or js or python or whatever but... it's easier to just let the LLM do it for me :|
      - satvikpendem 3y ago
        Or you could ask the AI for a script to convert it (whose resultant JSON would be free of hallucinations) and ask the AI to run the code, even, since I believe ChatGPT now has that ability, because I've definitely noticed that ChatGPT will sometimes mess up the data when converting from one format to another.
MrYellowP 3y ago
> information extraction task I do that with orca-mini-3b in ggml format and it's pretty good at it, at twice the speed. Of all the LLMs I've tried, this one gave me the best results. It just requires a properly written prompt.
- nsomaru 3y ago
  Could you elaborate on the prompting strategies you have used that are more effective?
dchuk 3y ago
Can you share an example of information extraction prompts? I specifically am interested in using LLMs as basically general purpose web scrapers that can take html and extract matching data per a prompt into structured json schema…do you think this is possible with llama 2?
behnamoh 3y ago
> ... and lower prices. Not sure about this. atm, the cost of any cloud GPU (spot or not) far exceeds the cost of OpenAI's API. I'd be glad to be proven wrong because I, too, want to run L2 (the 70b model). Also, buying a GPU, even 4090, is not feasible for most people. And it's not just about GPU—you'd have to build a PC for it to work, and there's the hidden maintenance cost of running desktop Linux (to use GPTQ for instance). It's not surprising that most users prefer someone else (OpenAI) to do it for them.
- jddj 3y ago
  I have to admit, I wouldn't have imagined even a few months ago that I'd be reading this comment. Sure, you can run something comparable to OpenAI's flagship product at home, but it's moderately expensive and slightly inconvenient so people will still pay for the convenience.
broast 3y ago
Any information about how long it takes to achieve a fine tune comparable to OpenAI's current fine tunes of ada models? On consumer hardware vs cloud? OpenAI's fine tune times are on the order of hours for tens of thousands of samples, but expensive. Any information on the effort and time involved in fine tuning compared to OpenAI's current process would be appreciated.

asabla 3y ago

> The enthusiasm around it reminds me of Javascript wars 10 years ago... so much energy! I kind of have the same feeling as well. With all this energy it's really hard to keep up with all new ideas, implementations, frameworks and services. Really excited for what this will bring us the next coming years

sva_ 3y ago
> With all this energy it's really hard to keep up with all new ideas, implementations, frameworks and services. The majority of them are mostly irrelevant. You just need to figure out which.

worldsavior 3y ago

It looks like it will always be a war like Android VS iOS only now it's with AI models.

pavlov 3y ago

> “The enthusiasm around it reminds me of JavaScript framework wars of 10 years ago” Hmm. If LLMs turned out like JS frameworks, that would mean that in ten years people will be saying: “Maybe we don’t really need all this expensive ceremony, honestly this could be done with vanilla if/else heuristics…?”

spacebanana7 3y ago
I can imagine a bloated world where 500B param models are used for tasks where 7B param modes perform adequately. At that time, there could be complaints on hacker news about messaging apps with autocomplete models that take up gigabytes.
- pavlov 3y ago
  I could see a world where people turn to an LLM for a task where the hand-rolled solution could be a simple state machine or some nested switch statements. The irony would be that the LLM could write you that code, but if you don’t know to ask…
  - falcor84 3y ago
    The real irony would be if the AI uses an LLM to write the code the second time it sees the same request repeated, deploys it to effortlessly deal with all future requests, and goes back to playing Quake while collecting the full paycheck.
- dmd 3y ago
  "How can I sum this column of numbers?" "IDK, throw it at the LLM"
  - azeirah 3y ago
    I do stuff like this sometimes when I have some csv or something and need it in JSON. Could easily be done with 1 line of bash or js or python or whatever but... it's easier to just let the LLM do it for me :|
    - satvikpendem 3y ago
      Or you could ask the AI for a script to convert it (whose resultant JSON would be free of hallucinations) and ask the AI to run the code, even, since I believe ChatGPT now has that ability, because I've definitely noticed that ChatGPT will sometimes mess up the data when converting from one format to another.

MrYellowP 3y ago

> information extraction task I do that with orca-mini-3b in ggml format and it's pretty good at it, at twice the speed. Of all the LLMs I've tried, this one gave me the best results. It just requires a properly written prompt.

nsomaru 3y ago
Could you elaborate on the prompting strategies you have used that are more effective?

dchuk 3y ago

Can you share an example of information extraction prompts? I specifically am interested in using LLMs as basically general purpose web scrapers that can take html and extract matching data per a prompt into structured json schema…do you think this is possible with llama 2?

behnamoh 3y ago

> ... and lower prices. Not sure about this. atm, the cost of any cloud GPU (spot or not) far exceeds the cost of OpenAI's API. I'd be glad to be proven wrong because I, too, want to run L2 (the 70b model). Also, buying a GPU, even 4090, is not feasible for most people. And it's not just about GPU—you'd have to build a PC for it to work, and there's the hidden maintenance cost of running desktop Linux (to use GPTQ for instance). It's not surprising that most users prefer someone else (OpenAI) to do it for them.

jddj 3y ago
I have to admit, I wouldn't have imagined even a few months ago that I'd be reading this comment. Sure, you can run something comparable to OpenAI's flagship product at home, but it's moderately expensive and slightly inconvenient so people will still pay for the convenience.

broast 3y ago

Any information about how long it takes to achieve a fine tune comparable to OpenAI's current fine tunes of ada models? On consumer hardware vs cloud? OpenAI's fine tune times are on the order of hours for tens of thousands of samples, but expensive. Any information on the effort and time involved in fine tuning compared to OpenAI's current process would be appreciated.