4 ms·

Using Topological Data Analysis to Solve Complex Problems (2013) [video]

42 points by espeed 11y ago

fitzwatermellow 11y ago
Interesting background on Ayasdi (literally, “to seek,” in Cherokee, perhaps sparking the newest trend in startup names deriving from indigenous languages) and its DARPA roots: http://bits.blogs.nytimes.com/2013/01/16/ayasdi-a-big-data-start-up-with-a-long-history/?_r=0 http://bits.blogs.nytimes.com/2013/01/16/ayasdi-a-big-data-s... And a link to founder and Stanford Math Prof. Gunnar Carlsson's seminal AMS paper on "Topology and Data": http://www.ams.org/journals/bull/2009-46-02/S0273-0979-09-01249-X/ http://www.ams.org/journals/bull/2009-46-02/S0273-0979-09-01...
irickt 11y ago
Papers are available on Ayasdi's site: http://www.ayasdi.com/approach/data-scientist/ http://www.ayasdi.com/approach/data-scientist/
- lsprack 11y ago
  There's also a very nice open-source R implementation of some of the TDA mapper functionality that I played with a while back: https://github.com/paultpearson/TDAmapper/ https://github.com/paultpearson/TDAmapper/
anthony_bak 11y ago
Hey. That's me. On hackernews :) Happy to answer questions here regarding TDA but am no longer with Ayasdi so won't comment on company specific stuff. The material in the video is dated but still relevant - I now have a better understanding of how to use "Shape" to improve Machine Learning models in an operational setting.
- gavazzy 11y ago
  Any new talks, videos, or papers to recommend?
  - anthony_bak 11y ago
    Sure. Gunnar's "Topology and Data" paper (already linked to by another poster) is the first place to go for an overview of TDA (although it's getting dated). A more updated version (but behind paywall) is "Topological pattern recognition for point cloud data" ( http://dx.doi.org/10.1017/S0962492914000051 http://dx.doi.org/10.1017/S0962492914000051 ) Robert Ghrist has another take on TDA ( https://www.math.upenn.edu/~ghrist/preprints.html https://www.math.upenn.edu/~ghrist/preprints.html ) that is more "engineering" focused and also uses a different mathematical tool set than the above (eg. Sheaves/Co-Sheaves). I particularly like the sensor coverage results. Rob Ghrist and John Harer ( http://fds.duke.edu/db/aas/math/faculty/john.harer/publications.html http://fds.duke.edu/db/aas/math/faculty/john.harer/publicati... )both have textbooks available if you want to get into the fundamentals in the field. Jose Perea has some nice results using ideas from TDA in a variety of contexts. eg texture classification ( https://www.math.msu.edu/user_content/docs/KleinBottleTextureAnalysis20150826163949977.pdf https://www.math.msu.edu/user_content/docs/KleinBottleTextur... ) and signal processing ( https://www.math.msu.edu/user_content/docs/Sw1Pes_Theory20150826163903619.pdf https://www.math.msu.edu/user_content/docs/Sw1Pes_Theory2015... ) Here's a talk I gave at ICERM last summer using persistent homology as a feature generating method for drug discovery: https://icerm.brown.edu/video_archive/#/play/726 https://icerm.brown.edu/video_archive/#/play/726 (slides available for download if you poke around on the icerm web site). This is my "part two" of the video linked to in the parent article: https://www.youtube.com/watch?v=3Z73Wd2T1xE https://www.youtube.com/watch?v=3Z73Wd2T1xE
    - fitzwatermellow 11y ago
      Thanks for the updated list, Anthony! On the practical side, what packages are you guys using? I'm familiar with JavaPlex for Matlab: http://appliedtopology.github.io/javaplex/ http://appliedtopology.github.io/javaplex/
      - anthony_bak 11y ago
        For Mapper consider using PythonMapper by Daniel Müllner ( http://danifold.net/mapper/ http://danifold.net/mapper/ ). The UI is touchy - I use it mostly via scripts only. Mapper is pretty simple to implement (just a series of well understood pieces - the magic is in the interpretation/understanding what you've done). As an example consider the kepler-mapper project ( https://github.com/MLWave/kepler-mapper https://github.com/MLWave/kepler-mapper ). More lines of code are used for calling out to the d3 visualization than implementing the core mapper algorithm. For my persistent homology calculations I always use Dionysus ( http://www.mrzv.org/software/dionysus/ http://www.mrzv.org/software/dionysus/ ). Rumor has it a much improved parallelized version will be released soon.

42 points by espeed 11y ago

Interesting background on Ayasdi (literally, “to seek,” in Cherokee, perhaps sparking the newest trend in startup names deriving from indigenous languages) and its DARPA roots: http://bits.blogs.nytimes.com/2013/01/16/ayasdi-a-big-data-start-up-with-a-long-history/?_r=0 http://bits.blogs.nytimes.com/2013/01/16/ayasdi-a-big-data-s... And a link to founder and Stanford Math Prof. Gunnar Carlsson's seminal AMS paper on "Topology and Data": http://www.ams.org/journals/bull/2009-46-02/S0273-0979-09-01249-X/ http://www.ams.org/journals/bull/2009-46-02/S0273-0979-09-01...

irickt 11y ago

Papers are available on Ayasdi's site: http://www.ayasdi.com/approach/data-scientist/ http://www.ayasdi.com/approach/data-scientist/

lsprack 11y ago
There's also a very nice open-source R implementation of some of the TDA mapper functionality that I played with a while back: https://github.com/paultpearson/TDAmapper/ https://github.com/paultpearson/TDAmapper/

anthony_bak 11y ago

Hey. That's me. On hackernews :) Happy to answer questions here regarding TDA but am no longer with Ayasdi so won't comment on company specific stuff. The material in the video is dated but still relevant - I now have a better understanding of how to use "Shape" to improve Machine Learning models in an operational setting.

gavazzy 11y ago
Any new talks, videos, or papers to recommend?
- anthony_bak 11y ago
  Sure. Gunnar's "Topology and Data" paper (already linked to by another poster) is the first place to go for an overview of TDA (although it's getting dated). A more updated version (but behind paywall) is "Topological pattern recognition for point cloud data" ( http://dx.doi.org/10.1017/S0962492914000051 http://dx.doi.org/10.1017/S0962492914000051 ) Robert Ghrist has another take on TDA ( https://www.math.upenn.edu/~ghrist/preprints.html https://www.math.upenn.edu/~ghrist/preprints.html ) that is more "engineering" focused and also uses a different mathematical tool set than the above (eg. Sheaves/Co-Sheaves). I particularly like the sensor coverage results. Rob Ghrist and John Harer ( http://fds.duke.edu/db/aas/math/faculty/john.harer/publications.html http://fds.duke.edu/db/aas/math/faculty/john.harer/publicati... )both have textbooks available if you want to get into the fundamentals in the field. Jose Perea has some nice results using ideas from TDA in a variety of contexts. eg texture classification ( https://www.math.msu.edu/user_content/docs/KleinBottleTextureAnalysis20150826163949977.pdf https://www.math.msu.edu/user_content/docs/KleinBottleTextur... ) and signal processing ( https://www.math.msu.edu/user_content/docs/Sw1Pes_Theory20150826163903619.pdf https://www.math.msu.edu/user_content/docs/Sw1Pes_Theory2015... ) Here's a talk I gave at ICERM last summer using persistent homology as a feature generating method for drug discovery: https://icerm.brown.edu/video_archive/#/play/726 https://icerm.brown.edu/video_archive/#/play/726 (slides available for download if you poke around on the icerm web site). This is my "part two" of the video linked to in the parent article: https://www.youtube.com/watch?v=3Z73Wd2T1xE https://www.youtube.com/watch?v=3Z73Wd2T1xE
  - fitzwatermellow 11y ago
    Thanks for the updated list, Anthony! On the practical side, what packages are you guys using? I'm familiar with JavaPlex for Matlab: http://appliedtopology.github.io/javaplex/ http://appliedtopology.github.io/javaplex/
    - anthony_bak 11y ago
      For Mapper consider using PythonMapper by Daniel Müllner ( http://danifold.net/mapper/ http://danifold.net/mapper/ ). The UI is touchy - I use it mostly via scripts only. Mapper is pretty simple to implement (just a series of well understood pieces - the magic is in the interpretation/understanding what you've done). As an example consider the kepler-mapper project ( https://github.com/MLWave/kepler-mapper https://github.com/MLWave/kepler-mapper ). More lines of code are used for calling out to the d3 visualization than implementing the core mapper algorithm. For my persistent homology calculations I always use Dionysus ( http://www.mrzv.org/software/dionysus/ http://www.mrzv.org/software/dionysus/ ). Rumor has it a much improved parallelized version will be released soon.