4 ms·

The title, as it stands, is trite and wrong. More about that a little later. The article on the other hand is a pleasant read. Topology is whatever little stru

by srean 1y ago

The title, as it stands, is trite and wrong. More about that a little later. The article on the other hand is a pleasant read. Topology is whatever little structure that remains in geometry after you throwaway distances, angles, orientations and all sorts of non tearing stretchings. It's that bare minimum that still remains valid after such violent deformations. While notion of topology is definitely useful in machine learning, -- scale, distance, angles etc., all usually provide lots of essential information about the data. If you want to distinguish between a tabby cat and a tiger it would be an act of stupidity to ignore scale. Topology is useful especially when you cannot trust lengths, distances angles and arbitrary deformations. That happens, but to claim deep learning is applied topology is absurd, almost stupid.

theahura 1y ago
> Topology is useful especially when you cannot trust lengths, distances angles and arbitrary deformations But...you can't. The input data lives on a manifold that you cannot 'trust'. It doesn't mean anything apriori that an image of a coca-cola can and an image of a stopsign live close to each other in pixel space. The neural network applies all of those violent transformations you are talking about
- srean 1y ago
  > But...you can't. Only in a desperate sales pitch or a desparate research grants. There are of course some situations were certain measurements are untrustworthy, but to claim that is the common case is very snake oily. When certain measurements become untrustworthy, that it does so only because of some unknown smooth transformation, is not very frequent (this is what purely topological methods will deal with). Random noise will also do that for you. Not disputing the fact that sometimes metrics cannot be trusted entirely, but to go to a topological approach seems extreme. One should use as much of the relevant non-topological information as possible. As the hackneyed example goes a topological methods would not be able to distinguish between a cup and a donut. For that you would need to trust non-topological features such as distances and angles. Deep learning methods can indeed differentiate between cop-nip and coffee mugs. BTW I am completely on-board with the idea that data often looks as if it has been sampled from an unknown, potentially smooth, possibly non-Euclidean manifold and then corrupted by noise. In such cases recovering that manifold from noisy data is a very worthy cause. In fact that is what most of your blogpost is about. But that's differential geometry and manifolds, they have structure far richer than a topology. For example they may have tangent planes, a Reimann metric or a symplectic form etc. A topological method would throw all of that away and focus on topology.
- kentuckyrobby 1y ago
  I don't think that was their point, I think their point was that neural networks 'create' their optimization space by using lengths, distances, and angles. You can't reframe it from a topological standpoint, otherwise optimization spaces of some similar neural networks on similar problems would topologically comparable, which is not true.
  - theahura 1y ago
    Well, sorta. There is some evidence to suggest that neural networks learn 'universal' features (cf Anthropic's circuits thread). But I'll openly admit to being out of my depth here, and maybe I just don't understand OPs point
throwawaymaths 1y ago
once you get into the nitty gritty, a lot of things that wouldn't matter if it were pure topology, do, like number of layers all the way to quantization/fp resolution
quantadev 1y ago
The word "topology" has a legitimate dictionary definition, that has none of the requirements that you're asserting. I think what you're missing is that it has two definitions.
- srean 1y ago
  In blog posts about specialised and technical topics it is expected that in-domain technical keywords that have long established definitions and meanings be used in the same technical sense. Otherwise it can become quite confusing. Gravity means gravity when we are talking Newtonian mechanics. Similarly, in math and ML 'topology' has a specific meaning.
  - quantadev 1y ago
    The word "topology" is quite commonly used in all kinds of books, papers, and technical materials any time they're discussing geometric characteristics of surfaces. The term is probably used 1000000 times more commonly in this more generic way than it's ever used in the strict pedantic way you're asserting that it must.
    - srean 1y ago
      Surfaces certainly have a topology (potentially more than one), surfaces are examples of one kind of a topological space, in fact the next interesting one after a curve. So I will not be surprised at all with co-occurrences of 'surface' and 'topology'. But surfaces and topologies mean different things. Dogs have fur. Dogs are an example of a furry animal. But dogs and furs are not the same thing although they may appear in the same text often. Topology is a traditional as well as an active branch of applied and pure mathematics, well, Physics too. It has tons of text books printed on it, has several active journals and conferences dealing with it. https://www.amazon.com/s?k=Topology&sprefix=topology+%2Caps%2C379 https://www.amazon.com/s?k=Topology&sprefix=topology+%2Caps%... Surprise, surprise ...not ...has an extensive Wikipedia page. https://en.m.wikipedia.org/wiki/Topology https://en.m.wikipedia.org/wiki/Topology Math magazines for high schoolers have articles on it. Colleges offer multiple courses on it. Some of those courses would be mandatory for a degree in even undergrad mathematics. If one wants to do graduate studies then one can do a Masters or a PhD in Topology, well in one of it's many branches. It's also not a new kid on the block. It goes back to ... analysis situs ... further back to Leibniz, although it began to crystalize formally after Poincare. If someone wants to use the phrase 'differential calculus' to mean something else in their love letters and sweet nothings, that's absolutely fine :) but in Maths (and Machine Learning, well, with quality of peer reviewing this might soon be iffy) it has a well established and unambiguous meaning. Note because of its shared beginning at the feet of Leibniz, comparing it with calculus is not an unfair comparison.

by srean 1y ago

theahura 1y ago

> Topology is useful especially when you cannot trust lengths, distances angles and arbitrary deformations But...you can't. The input data lives on a manifold that you cannot 'trust'. It doesn't mean anything apriori that an image of a coca-cola can and an image of a stopsign live close to each other in pixel space. The neural network applies all of those violent transformations you are talking about

srean 1y ago
> But...you can't. Only in a desperate sales pitch or a desparate research grants. There are of course some situations were certain measurements are untrustworthy, but to claim that is the common case is very snake oily. When certain measurements become untrustworthy, that it does so only because of some unknown smooth transformation, is not very frequent (this is what purely topological methods will deal with). Random noise will also do that for you. Not disputing the fact that sometimes metrics cannot be trusted entirely, but to go to a topological approach seems extreme. One should use as much of the relevant non-topological information as possible. As the hackneyed example goes a topological methods would not be able to distinguish between a cup and a donut. For that you would need to trust non-topological features such as distances and angles. Deep learning methods can indeed differentiate between cop-nip and coffee mugs. BTW I am completely on-board with the idea that data often looks as if it has been sampled from an unknown, potentially smooth, possibly non-Euclidean manifold and then corrupted by noise. In such cases recovering that manifold from noisy data is a very worthy cause. In fact that is what most of your blogpost is about. But that's differential geometry and manifolds, they have structure far richer than a topology. For example they may have tangent planes, a Reimann metric or a symplectic form etc. A topological method would throw all of that away and focus on topology.
kentuckyrobby 1y ago
I don't think that was their point, I think their point was that neural networks 'create' their optimization space by using lengths, distances, and angles. You can't reframe it from a topological standpoint, otherwise optimization spaces of some similar neural networks on similar problems would topologically comparable, which is not true.
- theahura 1y ago
  Well, sorta. There is some evidence to suggest that neural networks learn 'universal' features (cf Anthropic's circuits thread). But I'll openly admit to being out of my depth here, and maybe I just don't understand OPs point

throwawaymaths 1y ago

once you get into the nitty gritty, a lot of things that wouldn't matter if it were pure topology, do, like number of layers all the way to quantization/fp resolution

quantadev 1y ago

The word "topology" has a legitimate dictionary definition, that has none of the requirements that you're asserting. I think what you're missing is that it has two definitions.

srean 1y ago
In blog posts about specialised and technical topics it is expected that in-domain technical keywords that have long established definitions and meanings be used in the same technical sense. Otherwise it can become quite confusing. Gravity means gravity when we are talking Newtonian mechanics. Similarly, in math and ML 'topology' has a specific meaning.
- quantadev 1y ago
  The word "topology" is quite commonly used in all kinds of books, papers, and technical materials any time they're discussing geometric characteristics of surfaces. The term is probably used 1000000 times more commonly in this more generic way than it's ever used in the strict pedantic way you're asserting that it must.
  - srean 1y ago
    Surfaces certainly have a topology (potentially more than one), surfaces are examples of one kind of a topological space, in fact the next interesting one after a curve. So I will not be surprised at all with co-occurrences of 'surface' and 'topology'. But surfaces and topologies mean different things. Dogs have fur. Dogs are an example of a furry animal. But dogs and furs are not the same thing although they may appear in the same text often. Topology is a traditional as well as an active branch of applied and pure mathematics, well, Physics too. It has tons of text books printed on it, has several active journals and conferences dealing with it. https://www.amazon.com/s?k=Topology&sprefix=topology+%2Caps%2C379 https://www.amazon.com/s?k=Topology&sprefix=topology+%2Caps%... Surprise, surprise ...not ...has an extensive Wikipedia page. https://en.m.wikipedia.org/wiki/Topology https://en.m.wikipedia.org/wiki/Topology Math magazines for high schoolers have articles on it. Colleges offer multiple courses on it. Some of those courses would be mandatory for a degree in even undergrad mathematics. If one wants to do graduate studies then one can do a Masters or a PhD in Topology, well in one of it's many branches. It's also not a new kid on the block. It goes back to ... analysis situs ... further back to Leibniz, although it began to crystalize formally after Poincare. If someone wants to use the phrase 'differential calculus' to mean something else in their love letters and sweet nothings, that's absolutely fine :) but in Maths (and Machine Learning, well, with quality of peer reviewing this might soon be iffy) it has a well established and unambiguous meaning. Note because of its shared beginning at the feet of Leibniz, comparing it with calculus is not an unfair comparison.