pico_creator

8 ms·

91.
by pico_creator 3y ago
There is a reasonable argument for that (heard the idea go around between multiple AI engineers, that once you go past a certain scale, it does not matter for its evals) One of the biggest issue for testing all of this, is it takes a crap t
92.
by pico_creator 3y ago
One of the biggest issue with incredibly large context sizes is the lack of training / dataset meant specifically to use such large context sizes. And this apply to all models. So when a specific model now does badly even at 32k or 50k
93.
by pico_creator 3y ago
Completely the opposite. - it is NOT backed directly or owned by any VC funded company - it is 100% OSS driven by the community (Apache 2 license) - it’s currently the top OSS chat model that can be used commercially on the chatbot arena sc
94.
by pico_creator 3y ago
It has already been confirmed that with the right dataset we can scale it effectively from 2k to 4K, and 4K to 8k via fine tuning (you dun even need to train a new foundation model) We believe this can be done for 16k to way beyond 100k Res
95.
by pico_creator 3y ago
Yup. I’m all here for infinite scaling of context size
96.
by pico_creator 3y ago
AKA: no one here is incentivised to fight for bigger ownership, no promotion or KPI or pay was on the line
97.
by pico_creator 3y ago
RWKV has been running in public (relatively obscure to other models) for the past 2 years Mostly lead by a single person (blink). This community consist mostly of people outside the academia / big VC tech scene When eleutherAI offered
98.
by pico_creator 3y ago
This is straight to ARXIV (its not even the final copy yet)
99.
by pico_creator 3y ago
Hi Everyone, I'm a regular involved with the RWKV community. AMA, on RWKV, and I will do my best to answer them here for the next hour (one at a time) PS: you can find our discord here : https://discord.gg/qt9egFA7ve