3 ms·

Where can I read more about this field of research?

• latchkey 4y ago
  https://arxiv.org/ https://arxiv.org/ has a ton of papers on it.
  • elicksaur 4y ago
    Do you have some good search terms to get started down the rabbit hole?
    • franl 4y ago
      I too would like to go down this rabbit hole. I am going to poke around using the terms “distributed learning” and “federated learning” (They’re different areas, but somewhat related as far as I understand).
    • franl 4y ago
      This one is a few years old, but seems interesting: https://arxiv.org/abs/1802.05799v2 https://arxiv.org/abs/1802.05799v2
    • vagabund 4y ago
      Probably the biggest recent result: https://arxiv.org/abs/2209.04836 https://arxiv.org/abs/2209.04836 (author thread: https://twitter.com/SamuelAinsworth/status/1569719494645526529 https://twitter.com/SamuelAinsworth/status/15697194946455265...) See also: https://github.com/learning-at-home/hivemind https://github.com/learning-at-home/hivemind and more to OP's incentive structure: https://docs.bittensor.com/ https://docs.bittensor.com/ Latter two intend to beat latency with Mixture-of-Expert models (MoEs). If the results of the former hold, it shows that with a simple algorithmic transformation you can merge two independently trained models in weight-space and have performance functionally equivalent to a model trained monolithically.
• eternalban 4y ago
  A broad covering term is MLOps (like DevOps). https://en.wikipedia.org/wiki/MLOps https://en.wikipedia.org/wiki/MLOps Armed with that term, we get (haven’t read): Machine Learning Operations (MLOps): Overview, Definition, and Architecture https://arxiv.org/abs/2205.02302 https://arxiv.org/abs/2205.02302 [+ps] Better resource: https://ml-ops.org/ https://ml-ops.org/ Based on my (limited) exposure to date, there is tremendous opportunity for software engineers and architects to make impact in ML systems. There is a pronounced lack of seasoned engineering talent (outside of big players like DataBricks, et al) and this knowledge gap sits behind an experience curve that mere IQ can’t jump over. Our experience as software architects and engineers is very valuable. Know this and recognize the value you will bring to the table.
• joaogui1 4y ago
  This initiative is getting some pretty good practical results https://arxiv.org/abs/2209.01188 https://arxiv.org/abs/2209.01188, taking a look at their citations should lead you to other examples in the field