4 ms·

I think my point stands that the criticism on this thread is mostly a surface level reaction and hung up on meaningless slogans like "software 2.0" or "breakthr

I think my point stands that the criticism on this thread is mostly a surface level reaction and hung up on meaningless slogans like "software 2.0" or "breakthrough". You use of the word "seems" is very apt here. Have you considered that Google might have hired Lattner precisely because he is the founder of LLVM and Swift, and they hoped to leverage his organizational skills to jump start next generation tooling? We know google is heavily invested in llvm and C++, but dissatisfied with the direction C++ is heading [0]. They also are designing custom hardware like TPUs that isn't supported well by any current language. To me it seems like they are thinking a generation or two ahead with their tooling while the outside observers can't imagine anything beyond 80s era language design. [0] https://www.infoworld.com/article/3535795/c-plus-plus-proposal-dismisses-backward-compatibility.html https://www.infoworld.com/article/3535795/c-plus-plus-propos...

pjmlp 7y ago
> To me it seems like they are thinking a generation or two ahead with their tooling while the outside observers can't imagine anything beyond 80s era language design. Given the ML and Modula-3 influences in Swift, and the Xerox PARC work on Mesa/Cedar, it looks quite 80s era language design to me.
- DeathArrow 6y ago
  Swift inherits some APIs from Objective C. You have to use something like CFAbsoluteTimeGetCurrent while even in something not very modern like C# you would use DateTime.Now()
p1esk 7y ago
I'm a deep learning researcher. I have an 8 GPU server, and today I'm experimenting with deformable convolutions. Can you tell me why I should consider switching from Pytorch to Swift? Are there model implementations available in Swift and not available in Pytorch? Are these implementations significantly faster on 8 GPUs? Is it easier to implement complicated models in Swift than in Pytorch (after I spend a couple of months learning Swift)? Are you sure Google will not stop pushing "deep learning in Swift" after a year or two? If the answer to all these questions is "No", why should I care about this "new generation tooling"? EDIT: and I'm not really attached to Pytorch either. In the last 8 years I switched from cuda-convnet to Caffe, to Theano, to Tensorflow, to Pytorch, and now I'm curious about Jax. I have also written cuda kernels, and vectorized multithreaded neural network code in plain C (Cilk+ and AVX intrinsics) when it made sense to do so.
- flipgimble 7y ago
  I’m not telling you to switch. I don’t think the S4TF team is telling you to switch anytime soon. At best you might want to be aware and curious about why Google is investing in a statically typed language with built in differentiation, as opposed to python. Those that are interested in machine learning tooling or library development may see an opportunity to join early, especially when people have such irrational unfounded bias against a language, as evidenced by the hot takes in this thread. My personal opinion, that I don’t want to force on anyone, is that Swift as a technology is under-estimated outside of Apple and Google.
- mkolodny 7y ago
  I've taken Chris Lattner / Jeremy Howard's lessons on Swift for TensorFlow [0][1]. I'll try to paraphrase their answers to your questions: There aren't major benefits to using Swift4TensorFlow yet. But (most likely) there will be within the next year or two. You'll be able to do low level research (e.g. deformable convolutions) in a high level language (Swift), rather than needing to write CUDA, or waiting for PyTorch to write it for you. [0] https://course.fast.ai/videos/?lesson=13 https://course.fast.ai/videos/?lesson=13 [1] https://course.fast.ai/videos/?lesson=14 https://course.fast.ai/videos/?lesson=14
  - p1esk 7y ago
    You'll be able to do low level research (e.g. deformable convolutions) in a high level language (Swift), rather than needing to write CUDA Not sure I understand - will Swift automatically generate efficient GPU kernels for these low level ops, or will it be making calls to CuDNN, etc?
    - mkolodny 7y ago
      The first one. At least as of last year, Swift4TensorFlow's goal is to go from Swift -> XLA/MLIR -> GPU kernels.
      - p1esk 7y ago
BubRoss 7y ago
Your point doesn't stand because what you said was a defensive reaction to what you thought was criticism of swift. I think you have bought into the coolaide pretty hard here. Everything you are saying is a hopeful assumption of the future.

pjmlp 7y ago

> To me it seems like they are thinking a generation or two ahead with their tooling while the outside observers can't imagine anything beyond 80s era language design. Given the ML and Modula-3 influences in Swift, and the Xerox PARC work on Mesa/Cedar, it looks quite 80s era language design to me.

DeathArrow 6y ago
Swift inherits some APIs from Objective C. You have to use something like CFAbsoluteTimeGetCurrent while even in something not very modern like C# you would use DateTime.Now()

p1esk 7y ago

I'm a deep learning researcher. I have an 8 GPU server, and today I'm experimenting with deformable convolutions. Can you tell me why I should consider switching from Pytorch to Swift? Are there model implementations available in Swift and not available in Pytorch? Are these implementations significantly faster on 8 GPUs? Is it easier to implement complicated models in Swift than in Pytorch (after I spend a couple of months learning Swift)? Are you sure Google will not stop pushing "deep learning in Swift" after a year or two? If the answer to all these questions is "No", why should I care about this "new generation tooling"? EDIT: and I'm not really attached to Pytorch either. In the last 8 years I switched from cuda-convnet to Caffe, to Theano, to Tensorflow, to Pytorch, and now I'm curious about Jax. I have also written cuda kernels, and vectorized multithreaded neural network code in plain C (Cilk+ and AVX intrinsics) when it made sense to do so.

flipgimble 7y ago
I’m not telling you to switch. I don’t think the S4TF team is telling you to switch anytime soon. At best you might want to be aware and curious about why Google is investing in a statically typed language with built in differentiation, as opposed to python. Those that are interested in machine learning tooling or library development may see an opportunity to join early, especially when people have such irrational unfounded bias against a language, as evidenced by the hot takes in this thread. My personal opinion, that I don’t want to force on anyone, is that Swift as a technology is under-estimated outside of Apple and Google.
mkolodny 7y ago
I've taken Chris Lattner / Jeremy Howard's lessons on Swift for TensorFlow [0][1]. I'll try to paraphrase their answers to your questions: There aren't major benefits to using Swift4TensorFlow yet. But (most likely) there will be within the next year or two. You'll be able to do low level research (e.g. deformable convolutions) in a high level language (Swift), rather than needing to write CUDA, or waiting for PyTorch to write it for you. [0] https://course.fast.ai/videos/?lesson=13 https://course.fast.ai/videos/?lesson=13 [1] https://course.fast.ai/videos/?lesson=14 https://course.fast.ai/videos/?lesson=14
- p1esk 7y ago
  You'll be able to do low level research (e.g. deformable convolutions) in a high level language (Swift), rather than needing to write CUDA Not sure I understand - will Swift automatically generate efficient GPU kernels for these low level ops, or will it be making calls to CuDNN, etc?
  - mkolodny 7y ago
    The first one. At least as of last year, Swift4TensorFlow's goal is to go from Swift -> XLA/MLIR -> GPU kernels.
    - p1esk 7y ago

BubRoss 7y ago

Your point doesn't stand because what you said was a defensive reaction to what you thought was criticism of swift. I think you have bought into the coolaide pretty hard here. Everything you are saying is a hopeful assumption of the future.