3 ms·

Is this simply a consequence of exponential growth in CS publications driven by machine learning or is there something really going on here?

adamnemecek 4y ago
The field needs better foundations. CT is pretty good.
- KRAKRISMOTT 4y ago
  No. It won't make a significant (if any at all) difference to effectiveness. Rewriting Pytorch in Haskell won't magically get you AGI.
  - adamnemecek 4y ago
    It's not about rewriting things in Haskell but about using CT to reason about architectures.
  - solomonb 4y ago
    No one was suggesting rewriting anything in Haskell afaict..
  - bawolff 4y ago
    You wouldn't expect improved foundations to increase effectiveness in the short term. And AGI is totally irrelavent here.
- hgsgm 4y ago
  Why? How? The OP GitHub site doesn't promote any material that introduces the concepts at all. The "survey" paper at the top is nigh-impenetrable. I'm sure the category theorists are having fun modelling machine learning, but it doesn't show how machine learning benefits from the category theory.
  - adamnemecek 4y ago
    Residual connection serves as a feedback/trace a la trace in traced monoidal categories. That is one insight I have gleaned from CT.
    - resource0x 4y ago
      You beat me to this. Indeed, of all categories the monoidal ones are the most potent. Look how nicely they fit in crypto ledgers: https://www.cl.cam.ac.uk/events/syco/3/slides/Nester.pdf https://www.cl.cam.ac.uk/events/syco/3/slides/Nester.pdf \s
  - Yoric 4y ago
    Category Theory (just as all mathematical models for programming or subsets thereof) are building blocks for reasoning on what we build. Past applications of such mathematical models include: - programming languages with semantics that are better adapted to specific problems (e.g. Rust's ownership); - better compilers (see e.g. Haskell's supercompiler, which puts to shame `constexpr`-style features); - better static analyzers (e.g. better type systems, abstract interpretation, model checkers). In the case of Machine Learning, it might some day help us create Machine Learning that we can understand and trust better. Or it might fail. Or it might help us invent something different entirely, in 30 years.
    - riku_iki 4y ago
      why those approaches never picked up outside of some academia projects?..
      - Yoric 4y ago
        These days, Microsoft requires model-checking proofs before accepting new device drivers. That's their secret weapon that finally got (mostly) rid of the BSOD. I suspect that Apple is also using model-checking at various layers, but I have no proof :)
        4y ago
bgavran 4y ago
OP here. The exponential growth in CS publication is much faster. This repository is simply a testament that CT is slowly ramping up. It's meant to show what kind of expressive power and breadth current CT models have, which to my knowledge isn't something that's well-known outside of our niche community.
- bgavran 4y ago
  It's also meant to suggest where things are going (the kind of a chart I have in mind is this one https://twitter.com/bgavran3/status/1422206118688956420 https://twitter.com/bgavran3/status/1422206118688956420 ), though I understand this is something that deserves a much more substantial proof.
- moralestapia 4y ago
  >The exponential growth in CS publication is much faster. So ... yes?

adamnemecek 4y ago

The field needs better foundations. CT is pretty good.

KRAKRISMOTT 4y ago
No. It won't make a significant (if any at all) difference to effectiveness. Rewriting Pytorch in Haskell won't magically get you AGI.
- adamnemecek 4y ago
  It's not about rewriting things in Haskell but about using CT to reason about architectures.
- solomonb 4y ago
  No one was suggesting rewriting anything in Haskell afaict..
- bawolff 4y ago
  You wouldn't expect improved foundations to increase effectiveness in the short term. And AGI is totally irrelavent here.
hgsgm 4y ago
Why? How? The OP GitHub site doesn't promote any material that introduces the concepts at all. The "survey" paper at the top is nigh-impenetrable. I'm sure the category theorists are having fun modelling machine learning, but it doesn't show how machine learning benefits from the category theory.
- adamnemecek 4y ago
  Residual connection serves as a feedback/trace a la trace in traced monoidal categories. That is one insight I have gleaned from CT.
  - resource0x 4y ago
    You beat me to this. Indeed, of all categories the monoidal ones are the most potent. Look how nicely they fit in crypto ledgers: https://www.cl.cam.ac.uk/events/syco/3/slides/Nester.pdf https://www.cl.cam.ac.uk/events/syco/3/slides/Nester.pdf \s
- Yoric 4y ago
  Category Theory (just as all mathematical models for programming or subsets thereof) are building blocks for reasoning on what we build. Past applications of such mathematical models include: - programming languages with semantics that are better adapted to specific problems (e.g. Rust's ownership); - better compilers (see e.g. Haskell's supercompiler, which puts to shame `constexpr`-style features); - better static analyzers (e.g. better type systems, abstract interpretation, model checkers). In the case of Machine Learning, it might some day help us create Machine Learning that we can understand and trust better. Or it might fail. Or it might help us invent something different entirely, in 30 years.
  - riku_iki 4y ago
    why those approaches never picked up outside of some academia projects?..
    - Yoric 4y ago
      These days, Microsoft requires model-checking proofs before accepting new device drivers. That's their secret weapon that finally got (mostly) rid of the BSOD. I suspect that Apple is also using model-checking at various layers, but I have no proof :)
      - 4y ago

bgavran 4y ago

OP here. The exponential growth in CS publication is much faster. This repository is simply a testament that CT is slowly ramping up. It's meant to show what kind of expressive power and breadth current CT models have, which to my knowledge isn't something that's well-known outside of our niche community.

bgavran 4y ago
It's also meant to suggest where things are going (the kind of a chart I have in mind is this one https://twitter.com/bgavran3/status/1422206118688956420 https://twitter.com/bgavran3/status/1422206118688956420 ), though I understand this is something that deserves a much more substantial proof.
moralestapia 4y ago
>The exponential growth in CS publication is much faster. So ... yes?