3 ms·
S4TF lost out, not to Python, but to Python's AI/ML ecosystem -- people, projects, libraries, frameworks. Despite its many shortcomings, Python has become the
by cs702 6y ago
S4TF lost out, not to Python, but to Python's AI/ML ecosystem -- people, projects, libraries, frameworks.
Despite its many shortcomings, Python has become the lingua franca of AI and ML, to the point that whenever I come across a newly published AI or ML paper that I find interesting, I expect to be able to find code implementing it in Python.
For example, yesterday I saw a post on HN about approximating self-attention matrices in transformers, which have O(n²) computational cost, with a seemingly clever approach that has O(n) computational cost: https://news.ycombinator.com/item?id=26105455 https://news.ycombinator.com/item?id=26105455 . "Huh, that looks interesting," I thought, "let me see if I can find some code." Three clicks later, I found myself at https://github.com/mlpen/Nystromformer https://github.com/mlpen/Nystromformer -- the official implementation... in Python. A few moments later, I was playing with this thingamajiggy. No other language comes close to having this kind of ecosystem in AI and ML.
Julia still has a shot at becoming a viable alternative to Python, especially if the Julia developers can shorten the "time to first interactive plot" to make it as fast as Python's, but they face an uphill battle against such an entrenched ecosystem.
- onelovetwo 6y agoIts becoming like the same grasp js has on the web...which is sad to see.
- belval 6y agoThis is a false equivalence of the first order. What makes JS impossible to dethrone is that whatever replaces it would need to be supported by a critical mass of web browsers, which just won't happen. AI/ML on the other hand has no such restriction. Your new startup could build all their ML models in Java with DeepLearning4J and it would probably work fine. Python is used because it's easy to work with for experimenting, which is a big part of research and for production your can extract your weight and graph structure to run on TVM/TensorRT/ONNX. There is no equivalent vendor lock-in.
- cutler 6y agoDon't forget Web Assembly. That could easily dethrone JS but it will take time. All the major browsers have been evergreen for some time now so it's not like we're back in the early 2000s.
- arcturus17 6y agoWell you could also build your web startup in a compile-to-JS language and barely ever touch JS, and you’d also be fine, right? (Maybe in rare cases for interop, debug, etc) At any rate you’re right - the dominance of each language for their respective use cases (JS-Web / Py-ML) emanates from different sources.
- lostmsu 6y agoNystromformer attention layer is actually pretty simple. You can reimplement it in a few minutes.
- cs702 6y agoSure, but isn't it nice that you don't even have to read the paper to play with the full model, to see if the claims hold up, using the same Python tools you and everyone else uses?
- lostmsu 6y agoFair, but I would not call that "ecosystem".
- cs702 6y agoIt was an example. Please don't argue against a strawman.
- dnautics 6y agoTo be honest that is not really true for many ML implementations from papers. Often, I will pull a repo and it will be half-baked, or, only works on python 2.4, or have a broken dependency on something or another (zmq was common issue for a hot sec), or not publish weights, or be in tf when I want torch, or be in torch when I want tf, etc.
- cs702 6y agoYes, I agree: a lot of code is indeed half-baked! But good papers tend to have better-than-average code, and ~100% of the time, the code is in Python, not some other language.
- deleted 6y ago[deleted]