3 ms·
Somehow not surprised Bellard is also going to make an impact in AI/LLMs
by tarruda 4y ago
Somehow not surprised Bellard is also going to make an impact in AI/LLMs
- synctext 4y ago> going to make an impact in AI/LLMs This seems to be made for mass parallelism. Question: Would this scale to 100 million of collaborating 4G/5G smartphones? We have operational federated learning code and decentralised learning code (trivial stochastic gradient descent). Application: simple learn-to-Rank based on ClickLogs like its 2005 [1]. Then you have a true decentralised Google. Would love to link RWKV to other pure decentralised tech. In the past we have build the first self-compiling Android app and first Android-to-Android P2P overlay network. Without any helper peers for carrier-grade NAT puncturing. My university systems lab lacks the size to keep up with the recent pace of innovation. [1] https://grouplens.org/beyond2005/full/pouwelse.pdf https://grouplens.org/beyond2005/full/pouwelse.pdf
- mdp2021 4y ago> Somehow not surprised Bellard is also going to make an impact in AI[... ] He also wrote an engine: "LibNC: C Library for Tensor Manipulation" https://bellard.org/libnc/ https://bellard.org/libnc/ > [ ...]/LLMs That could be a wish come true. Edit: in fact, already one of his marks can be seen: «Larger models work optimally on lower cost GPUs (e.g. RTX 3090, RTX A6000) thanks to efficient quantization»