7 ms·

Generally, I tend to care more about math if it relates to my specific interests. I'm in ML and robotics so I've found it useful to understand algebra, topology

Generally, I tend to care more about math if it relates to my specific interests. I'm in ML and robotics so I've found it useful to understand algebra, topology, real analysis. I've tried to get into Category theory a few times but couldn't keep my motivation high enough once things got too technical. Anyone know how category theory could be useful for someone in ML or robotics?

vivekseth 7y ago
Recently, I've been into learning about CRDTs a class of data structures which can be edited independently and merged without conflicts. They can be used to build experiences similar to Google Docs (although Gdocs uses a different approach called OT). I'm interested in learning Category/Set Theory to better understand the underlying math of how they work. I have very little experience with ML/Robotics, but CRDTs might be useful for distributed learning, or coordination between independent robots.
- formalsystem 7y ago
  It's funny you mention them, I've been thinking a lot about CRDTs for online multiplayer games where you have a bunch of concurrent players changing a game state.
- Twisol 7y ago
  I've been taking a more domain-theoretic approach to understanding CRDTs(focusing more on the semantics than the syntax) but it appears that domain theory does have some connections to category theory: https://ncatlab.org/nlab/show/domain+theory https://ncatlab.org/nlab/show/domain+theory
- adamnemecek 7y ago
  What class was this if I may ask?
  - vivekseth 7y ago
    When I said "class", I really just meant "type". CRDTs are relatively new so there may not be many classes yet that teach them. If you want to learn about them, I recommend you first watch this video: https://www.youtube.com/watch?v=OOlnp2bZVRs&list=WL&index=56&t=0s https://www.youtube.com/watch?v=OOlnp2bZVRs&list=WL&index=56... and then read this paper: https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf . The video explains a lot of the math in a very easy to digest way and will make the paper much easier to understand.
    - adamnemecek 7y ago
      I’m sorry, I misread. I was aware of CRDTs and I thought that they were taught in some class.
apex3stoker 7y ago
My understanding is that category theory can be considered as abstract function theory. If we can prove that a function (e.g. Haskell compiler) is equivalent to automatic differentiation with some transformation, according to category theory, we can write Haskell code to train neural network, instead of writing extra graph building code in, say, Tensorflow. This paper basically shows this. http://conal.net/papers/essence-of-ad/essence-of-ad-icfp.pdf http://conal.net/papers/essence-of-ad/essence-of-ad-icfp.pdf.
- formalsystem 7y ago
  You may be interested in this paper too https://arxiv.org/abs/1711.10455 https://arxiv.org/abs/1711.10455 I think given a language with AD you already don't need to have a separate language for graph building and my understanding is that's one of the goals for the Julia team and I've seen other nice looking packages for AD in other languages * https://fluxml.ai/2019/02/07/what-is-differentiable-programming.html https://fluxml.ai/2019/02/07/what-is-differentiable-programm... * http://diffsharp.github.io/DiffSharp/examples-inversekinematics.html http://diffsharp.github.io/DiffSharp/examples-inversekinemat... I'm also very excited about being able to design neural nets using ideas from functional programming. I love the Keras API but I don't think it's going to be the obvious API a couple of years down the line.
- eru 7y ago
  And for a completely different application of differentiation to programming, see http://strictlypositive.org/diff.pdf http://strictlypositive.org/diff.pdf
mathgenius 7y ago
Have you seen Robert Ghrist's book, "Elementary applied topology" ? There's plenty of category theory in there, even though it is mostly implicit. (And applications to ML and robotics.) https://www.math.upenn.edu/~ghrist/notes.html https://www.math.upenn.edu/~ghrist/notes.html
- formalsystem 7y ago
  Yes! I skimmed it and really enjoyed it although I mostly remember seeing algebraic topology and not so much category theory, will give it a closer look.
pgustafs 7y ago
I'm currently working on developing a double category of hybrid systems with applications to parallel composition and model reduction in robotics. We should have a math paper up on the arxiv within a month, and a robotics paper up shortly thereafter.
- formalsystem 7y ago
  Looking forward to it! And if there's a preprint you're comfortable sharing please let me know!
wjn0 7y ago
Check out UMAP [1]. It's an alternative to t-SNE for dimension reduction that's grounded in topological category theory. [1] https://arxiv.org/abs/1802.03426 https://arxiv.org/abs/1802.03426
KirinDave 7y ago
The structure of various ML primitives can be considered and under CT. Doubly so since in many cases you can in fact "go backwards" through a ML structure in a way normal functions can't, meaning that categorical duals might have more useful interpretations in your world. So if anything, it might be MORE applicable for composing ML networks.
- eru 7y ago
  Yes. And other bits commonly associated with functional programming should also work together really well with ML. After all, most ML models are explicitly thought about as a composition of layers (ie functions). But so far, the best ML libraries can be found in impure / imperative languages like Python.
- formalsystem 7y ago
  Is there any paper or book you like that covers this topic in more depth?
  - KirinDave 7y ago
    Just look for introductions to duality in math and category theory. It's not the sort of subject that'll be broken out individually, but rather an assumed consequence.

vivekseth 7y ago

Recently, I've been into learning about CRDTs a class of data structures which can be edited independently and merged without conflicts. They can be used to build experiences similar to Google Docs (although Gdocs uses a different approach called OT). I'm interested in learning Category/Set Theory to better understand the underlying math of how they work. I have very little experience with ML/Robotics, but CRDTs might be useful for distributed learning, or coordination between independent robots.

formalsystem 7y ago
It's funny you mention them, I've been thinking a lot about CRDTs for online multiplayer games where you have a bunch of concurrent players changing a game state.
Twisol 7y ago
I've been taking a more domain-theoretic approach to understanding CRDTs(focusing more on the semantics than the syntax) but it appears that domain theory does have some connections to category theory: https://ncatlab.org/nlab/show/domain+theory https://ncatlab.org/nlab/show/domain+theory
adamnemecek 7y ago
What class was this if I may ask?
- vivekseth 7y ago
  When I said "class", I really just meant "type". CRDTs are relatively new so there may not be many classes yet that teach them. If you want to learn about them, I recommend you first watch this video: https://www.youtube.com/watch?v=OOlnp2bZVRs&list=WL&index=56&t=0s https://www.youtube.com/watch?v=OOlnp2bZVRs&list=WL&index=56... and then read this paper: https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf . The video explains a lot of the math in a very easy to digest way and will make the paper much easier to understand.
  - adamnemecek 7y ago
    I’m sorry, I misread. I was aware of CRDTs and I thought that they were taught in some class.

apex3stoker 7y ago

My understanding is that category theory can be considered as abstract function theory. If we can prove that a function (e.g. Haskell compiler) is equivalent to automatic differentiation with some transformation, according to category theory, we can write Haskell code to train neural network, instead of writing extra graph building code in, say, Tensorflow. This paper basically shows this. http://conal.net/papers/essence-of-ad/essence-of-ad-icfp.pdf http://conal.net/papers/essence-of-ad/essence-of-ad-icfp.pdf.

formalsystem 7y ago
You may be interested in this paper too https://arxiv.org/abs/1711.10455 https://arxiv.org/abs/1711.10455 I think given a language with AD you already don't need to have a separate language for graph building and my understanding is that's one of the goals for the Julia team and I've seen other nice looking packages for AD in other languages * https://fluxml.ai/2019/02/07/what-is-differentiable-programming.html https://fluxml.ai/2019/02/07/what-is-differentiable-programm... * http://diffsharp.github.io/DiffSharp/examples-inversekinematics.html http://diffsharp.github.io/DiffSharp/examples-inversekinemat... I'm also very excited about being able to design neural nets using ideas from functional programming. I love the Keras API but I don't think it's going to be the obvious API a couple of years down the line.
eru 7y ago
And for a completely different application of differentiation to programming, see http://strictlypositive.org/diff.pdf http://strictlypositive.org/diff.pdf

mathgenius 7y ago

Have you seen Robert Ghrist's book, "Elementary applied topology" ? There's plenty of category theory in there, even though it is mostly implicit. (And applications to ML and robotics.) https://www.math.upenn.edu/~ghrist/notes.html https://www.math.upenn.edu/~ghrist/notes.html

formalsystem 7y ago
Yes! I skimmed it and really enjoyed it although I mostly remember seeing algebraic topology and not so much category theory, will give it a closer look.

pgustafs 7y ago

I'm currently working on developing a double category of hybrid systems with applications to parallel composition and model reduction in robotics. We should have a math paper up on the arxiv within a month, and a robotics paper up shortly thereafter.

formalsystem 7y ago
Looking forward to it! And if there's a preprint you're comfortable sharing please let me know!

wjn0 7y ago

Check out UMAP [1]. It's an alternative to t-SNE for dimension reduction that's grounded in topological category theory. [1] https://arxiv.org/abs/1802.03426 https://arxiv.org/abs/1802.03426

KirinDave 7y ago

The structure of various ML primitives can be considered and under CT. Doubly so since in many cases you can in fact "go backwards" through a ML structure in a way normal functions can't, meaning that categorical duals might have more useful interpretations in your world. So if anything, it might be MORE applicable for composing ML networks.

eru 7y ago
Yes. And other bits commonly associated with functional programming should also work together really well with ML. After all, most ML models are explicitly thought about as a composition of layers (ie functions). But so far, the best ML libraries can be found in impure / imperative languages like Python.
formalsystem 7y ago
Is there any paper or book you like that covers this topic in more depth?
- KirinDave 7y ago
  Just look for introductions to duality in math and category theory. It's not the sort of subject that'll be broken out individually, but rather an assumed consequence.