3 ms·

>We are waiting for huggingface to officially support the llama models (i.e. this PR to be merged) before we release a stable version of the finetuning code. C

>We are waiting for huggingface to officially support the llama models (i.e. this PR to be merged) before we release a stable version of the finetuning code. Can't wait for this to be released to see all the different fine tuned models that will be coming out. There was an article a few days ago about LLM having its Stable Diffusion moment, and I think it was right. We went from LLama release > llama.cpp > Dalai and now an instruction tuned model in under 2 weeks.

wkat4242 4y ago
Yeah this whole industry seems to suddenly have moved from the pace of a pebble creek to a waterfall. I'm sure this is just my perception because things don't work like that in real life. Is it really just a ton of slowly developed things happening to mature close together?
- RcouF1uZ4gsC 4y ago
  > Is it really just a ton of slowly developed things happening to mature close together? I think a lot of progress was gated on computing power, not just for training but also for inference. Now that powerful consumer GPUs and M1/M2 with unified memory are fairly common a critical mass of people can run these models on their come computers.
- layer8 4y ago
  Progress almost always comes in phases and jumps, this is not really surprising.
- whimsicalism 4y ago
  > Is it really just a ton of slowly developed things happening to mature close together Yes. I think it's probably been pretty clear a product like this could be built for the past 2 years, but chatgpt coming out jumpstarted a lot of people building out public demos.
- nestorD 4y ago
  > Is it really just a ton of slowly developed things happening to mature close together? It is a bit of both. Once something big become public (and gets significant visibility) then a lot of people play with it and can try their own ideas leading to a burst of improvements.
- yieldcrv 4y ago
  ChatGPT was the catalyst that there is market demand and what to do, that's literally it. Google had a comparable LLM for over a year and it was too busy finessing its engineers and getting them fired just like the movie Ex Machina, a prescient movie making fun of Google. The organization just sat on it. Plenty of others were in the same place, no imagined product market fit.
shostack 4y ago
I need a chat bot baked into a news app that can summarize the latest developments for me. It's too hard to keep up on my own!
- int_19h 4y ago
  Bing kinda already does that, esp. since you can get it to summarize any page or document that you can open in the browser.
lxe 4y ago
Check this out: https://github.com/facebookresearch/llama/issues/169 https://github.com/facebookresearch/llama/issues/169 Also I'm working on a very hacky way to tune with the hf ecosystem as my very first attempt to do deepspeed/accelerate/transformers https://github.com/lxe/llama-tune https://github.com/lxe/llama-tune

wkat4242 4y ago

Yeah this whole industry seems to suddenly have moved from the pace of a pebble creek to a waterfall. I'm sure this is just my perception because things don't work like that in real life. Is it really just a ton of slowly developed things happening to mature close together?

RcouF1uZ4gsC 4y ago
> Is it really just a ton of slowly developed things happening to mature close together? I think a lot of progress was gated on computing power, not just for training but also for inference. Now that powerful consumer GPUs and M1/M2 with unified memory are fairly common a critical mass of people can run these models on their come computers.
layer8 4y ago
Progress almost always comes in phases and jumps, this is not really surprising.
whimsicalism 4y ago
> Is it really just a ton of slowly developed things happening to mature close together Yes. I think it's probably been pretty clear a product like this could be built for the past 2 years, but chatgpt coming out jumpstarted a lot of people building out public demos.
nestorD 4y ago
> Is it really just a ton of slowly developed things happening to mature close together? It is a bit of both. Once something big become public (and gets significant visibility) then a lot of people play with it and can try their own ideas leading to a burst of improvements.
yieldcrv 4y ago
ChatGPT was the catalyst that there is market demand and what to do, that's literally it. Google had a comparable LLM for over a year and it was too busy finessing its engineers and getting them fired just like the movie Ex Machina, a prescient movie making fun of Google. The organization just sat on it. Plenty of others were in the same place, no imagined product market fit.

shostack 4y ago

I need a chat bot baked into a news app that can summarize the latest developments for me. It's too hard to keep up on my own!

int_19h 4y ago
Bing kinda already does that, esp. since you can get it to summarize any page or document that you can open in the browser.

lxe 4y ago

Check this out: https://github.com/facebookresearch/llama/issues/169 https://github.com/facebookresearch/llama/issues/169 Also I'm working on a very hacky way to tune with the hf ecosystem as my very first attempt to do deepspeed/accelerate/transformers https://github.com/lxe/llama-tune https://github.com/lxe/llama-tune