4 ms·

RWKV.cpp is now being deployed with the latest Windows 11 system

27 points by xnhbx 2y ago

ziddoap 2y ago
More info than the twitter post: https://blog.rwkv.com/p/rwkvcpp-shipping-to-half-a-billion https://blog.rwkv.com/p/rwkvcpp-shipping-to-half-a-billion No idea what this thing is, but it appears to be some sort of AI-related thing: >While it’s unclear what Microsoft is specifically using our models for, it is believed, this is in preparation for local Co-pilot running with on-device models
notamy 2y ago
I didn’t know what RWKV.cpp was, so hopefully this helps others: https://github.com/RWKV/rwkv.cpp https://github.com/RWKV/rwkv.cpp > INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
vessenes 2y ago
For those wondering, the RWKV architecture is an alternative to a transformer arch, and has the nice property of allowing very long inputs. Speculation here that it might be for code assistance would make sense. Early versions of RWKV that I played with took a long time to tokenize input strings, but generation was quick. I could imagine engineering finding a good fit with a codebase that’s going to be mostly static while an engineer is editing only parts of it.
deleted 2y ago
[deleted]
pico_creator 2y ago
Hey there, Twitter author / guy from the RWKV team. Some updates: Its also arriving to windows 10 (so that's 1.5B deploys) Our main guess would be the copilot beta features being tested - local copilot - local memory recall And it makes sense, especially for our smaller models - we support 100+ languages - we are extremely low in energy cost If anyone has a machine with local copilot / memory recall enabled, please reach out to me on my twitter @picocreator - I want to decompile and trace this down =)
methou 2y ago
“Silently overnight, it’s everywhere, in every Windows 10 and 11 PC.” So they are willing to deploy AI BS to an “old” system (win10), rather than the amd performance fix users desperately need.

4 ms·

RWKV.cpp is now being deployed with the latest Windows 11 system

27 points by xnhbx 2y ago

ziddoap 2y ago
More info than the twitter post: https://blog.rwkv.com/p/rwkvcpp-shipping-to-half-a-billion https://blog.rwkv.com/p/rwkvcpp-shipping-to-half-a-billion No idea what this thing is, but it appears to be some sort of AI-related thing: >While it’s unclear what Microsoft is specifically using our models for, it is believed, this is in preparation for local Co-pilot running with on-device models
notamy 2y ago
I didn’t know what RWKV.cpp was, so hopefully this helps others: https://github.com/RWKV/rwkv.cpp https://github.com/RWKV/rwkv.cpp > INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
vessenes 2y ago
For those wondering, the RWKV architecture is an alternative to a transformer arch, and has the nice property of allowing very long inputs. Speculation here that it might be for code assistance would make sense. Early versions of RWKV that I played with took a long time to tokenize input strings, but generation was quick. I could imagine engineering finding a good fit with a codebase that’s going to be mostly static while an engineer is editing only parts of it.
deleted 2y ago
[deleted]
pico_creator 2y ago
Hey there, Twitter author / guy from the RWKV team. Some updates: Its also arriving to windows 10 (so that's 1.5B deploys) Our main guess would be the copilot beta features being tested - local copilot - local memory recall And it makes sense, especially for our smaller models - we support 100+ languages - we are extremely low in energy cost If anyone has a machine with local copilot / memory recall enabled, please reach out to me on my twitter @picocreator - I want to decompile and trace this down =)
methou 2y ago
“Silently overnight, it’s everywhere, in every Windows 10 and 11 PC.” So they are willing to deploy AI BS to an “old” system (win10), rather than the amd performance fix users desperately need.