5 ms·

Does no one build their own ml algos anymore? I don't understand the need for pytorch and tensor flow. I honestly thought tensor flow was nothing but a teaching

Does no one build their own ml algos anymore? I don't understand the need for pytorch and tensor flow. I honestly thought tensor flow was nothing but a teaching thing for undergrads

kiloreux 7y ago
Not all of us need to build their own ML algos. Just in the same way that not all of us need to build their sorting libraries or data structures. Some people are specialized in this to develop and do research. While other software engineers just want something they can use without much hassle and just a superficial understanding.
- morningseagulls 7y ago
  >Not all of us need to build their own ML algos. Just in the same way that not all of us need to build their sorting libraries or data structures. And yet they love to ask you to do exactly that at technical interviews... coming up next: what ML algos you need to know to ace that interview.
- amelius 7y ago
  And another reason is standardization. It's a lot easier to use or tweak any given network if it is implemented in the same framework.
- dlphn___xyz 7y ago
  good luck trying to land a job using ml with only ‘superficial understanding’
falkaer 7y ago
They're frameworks which implement high performance tools commonly used in ml problems like tensor operations, automatic differentiation, various gradient descent optimisers, and also neural network building blocks
ibab 7y ago
Do you also write your own automatic differentiation tools? Using libraries like TF and PyTorch makes sense if you use neural networks because they provide automatic differentiation (who wants to write out their gradients by hand?) and standard neural network components. Edit: If your algorithm is not using neural networks, then libraries like TF may or may not be a good fit, it depends on the algorithm. Writing custom low-level code can still make sense in those cases.
- improbable22 7y ago
  People do write their own AD tools... in a day, sometimes! http://blog.rogerluo.me/2018/10/23/write-an-ad-in-one-day/ http://blog.rogerluo.me/2018/10/23/write-an-ad-in-one-day/ http://blog.rogerluo.me/2019/07/27/yassad/ http://blog.rogerluo.me/2019/07/27/yassad/ Although the endpoint is likely to be a better understanding of the choices made by a mature implementation, and of the work involved in fixing up edge cases.
ghaff 7y ago
Do you write your own crypto libraries too?
- morningseagulls 7y ago
  >Do you write your own crypto libraries too? Some people do. It's a good challenge.[0] [0] https://cryptopals.com/ https://cryptopals.com/
  - detaro 7y ago
    But the OP clearly didn't ask about doing it as a challenge or to understand how it works (which is what cryptopals is about), but actual usage.
    - morningseagulls 7y ago
      >But the OP clearly didn't ask about doing it as a challenge or to understand how it works (which is what cryptopals is about), but actual usage. But the parent of the comment I was replying to clearly had the former in mind, as a subsequent comment showed.[0] [0] https://news.ycombinator.com/item?id=21240429 https://news.ycombinator.com/item?id=21240429
  - dahart 7y ago
    To be fair, the main thing you learn doing the cryptopals challenge is to not write your own crypto. I had a lesson in writing crypto once, when I made what I thought was a good enough secret mixing procedure to encode some data I wanted to email outside of a company that didn’t allow web access. (Long time ago, circa 2000). It all looked undecipherable and I sent most of the data before I discovered that strings of binary zero were leaking my secret key. Oops, pretty stupid.
    - morningseagulls 7y ago
      >To be fair, the main thing you learn doing the cryptopals challenge is to not write your own crypto. D'oh! Good point though lol.
mrfox321 7y ago
Why would you write your own AutoDiff if you do not have to?
tsm212 7y ago
This type of reasoning can be extended to any high-level tool. " Does no one writes there own OS. I don't understand the need for Linux or windows. I honestly thought windows or linux was nothing but a tool for undergrads to use Excel or host a WordPress site". And this is not a caricature of your argument. There is a lot of stuff under the hood that Tensorflow or Pytorch implement for a programmer. So much so that people have written wrapper for using TF or Pytorch to even further abstract the working of the library. Implementing deeplearning architecture is less of a science and more of a "let me try this or that" and iterating ideas quickly if of the utmost importance. Also, I can implement a neural network in C (CUDA) although not the auto diff part, but I could if given time to research) but if I started implementing my own library, it would take an order (or even more) of magnitude more time to do the stuff I do daily. We don't need to reinvent the wheel here guys.
- __Asturias__ 7y ago
  That's what I'm getting at, the stuff under the hood is what's important, devil in the details and all that. I'm also a quant so every ml algo needs to be tailored so idk
TruckingThrow 7y ago
I can actually speak to this because I was involved with maintenance on a consultant's neural net built in raw java. What I have to say is this: please don't build your own.
mlevental 7y ago
can we see some of the "ml algos" you've built? in particular your autodiff engine
- __Asturias__ 7y ago
  They're all owned by my firm lol, I can't share anything like that
  - mlevental 7y ago
    https://media.giphy.com/media/gBpY4p7bbhsiI/giphy.gif https://media.giphy.com/media/gBpY4p7bbhsiI/giphy.gif
et2o 7y ago
childish troll attempt
halfarmbandit 7y ago
That’s quite naive... I am sure you could write stuff like Diffentiable Processors or the like from scratch with numpy but if you respect yourself and your time, you won’t. Complicated architectures are orders of magnitude harder than writing feed forward networks from scratch. For example, see the Merlin paper.

Does no one build their own ml algos anymore? I don't understand the need for pytorch and tensor flow. I honestly thought tensor flow was nothing but a teaching thing for undergrads

kiloreux 7y ago

Not all of us need to build their own ML algos. Just in the same way that not all of us need to build their sorting libraries or data structures. Some people are specialized in this to develop and do research. While other software engineers just want something they can use without much hassle and just a superficial understanding.

morningseagulls 7y ago
>Not all of us need to build their own ML algos. Just in the same way that not all of us need to build their sorting libraries or data structures. And yet they love to ask you to do exactly that at technical interviews... coming up next: what ML algos you need to know to ace that interview.
amelius 7y ago
And another reason is standardization. It's a lot easier to use or tweak any given network if it is implemented in the same framework.
dlphn___xyz 7y ago
good luck trying to land a job using ml with only ‘superficial understanding’

falkaer 7y ago

They're frameworks which implement high performance tools commonly used in ml problems like tensor operations, automatic differentiation, various gradient descent optimisers, and also neural network building blocks

ibab 7y ago

Do you also write your own automatic differentiation tools? Using libraries like TF and PyTorch makes sense if you use neural networks because they provide automatic differentiation (who wants to write out their gradients by hand?) and standard neural network components. Edit: If your algorithm is not using neural networks, then libraries like TF may or may not be a good fit, it depends on the algorithm. Writing custom low-level code can still make sense in those cases.

improbable22 7y ago
People do write their own AD tools... in a day, sometimes! http://blog.rogerluo.me/2018/10/23/write-an-ad-in-one-day/ http://blog.rogerluo.me/2018/10/23/write-an-ad-in-one-day/ http://blog.rogerluo.me/2019/07/27/yassad/ http://blog.rogerluo.me/2019/07/27/yassad/ Although the endpoint is likely to be a better understanding of the choices made by a mature implementation, and of the work involved in fixing up edge cases.

ghaff 7y ago

Do you write your own crypto libraries too?

morningseagulls 7y ago
>Do you write your own crypto libraries too? Some people do. It's a good challenge.[0] [0] https://cryptopals.com/ https://cryptopals.com/
- detaro 7y ago
  But the OP clearly didn't ask about doing it as a challenge or to understand how it works (which is what cryptopals is about), but actual usage.
  - morningseagulls 7y ago
    >But the OP clearly didn't ask about doing it as a challenge or to understand how it works (which is what cryptopals is about), but actual usage. But the parent of the comment I was replying to clearly had the former in mind, as a subsequent comment showed.[0] [0] https://news.ycombinator.com/item?id=21240429 https://news.ycombinator.com/item?id=21240429
- dahart 7y ago
  To be fair, the main thing you learn doing the cryptopals challenge is to not write your own crypto. I had a lesson in writing crypto once, when I made what I thought was a good enough secret mixing procedure to encode some data I wanted to email outside of a company that didn’t allow web access. (Long time ago, circa 2000). It all looked undecipherable and I sent most of the data before I discovered that strings of binary zero were leaking my secret key. Oops, pretty stupid.
  - morningseagulls 7y ago
    >To be fair, the main thing you learn doing the cryptopals challenge is to not write your own crypto. D'oh! Good point though lol.

mrfox321 7y ago

Why would you write your own AutoDiff if you do not have to?

tsm212 7y ago

This type of reasoning can be extended to any high-level tool. " Does no one writes there own OS. I don't understand the need for Linux or windows. I honestly thought windows or linux was nothing but a tool for undergrads to use Excel or host a WordPress site". And this is not a caricature of your argument. There is a lot of stuff under the hood that Tensorflow or Pytorch implement for a programmer. So much so that people have written wrapper for using TF or Pytorch to even further abstract the working of the library. Implementing deeplearning architecture is less of a science and more of a "let me try this or that" and iterating ideas quickly if of the utmost importance. Also, I can implement a neural network in C (CUDA) although not the auto diff part, but I could if given time to research) but if I started implementing my own library, it would take an order (or even more) of magnitude more time to do the stuff I do daily. We don't need to reinvent the wheel here guys.

__Asturias__ 7y ago
That's what I'm getting at, the stuff under the hood is what's important, devil in the details and all that. I'm also a quant so every ml algo needs to be tailored so idk

TruckingThrow 7y ago

I can actually speak to this because I was involved with maintenance on a consultant's neural net built in raw java. What I have to say is this: please don't build your own.

mlevental 7y ago

can we see some of the "ml algos" you've built? in particular your autodiff engine

__Asturias__ 7y ago
They're all owned by my firm lol, I can't share anything like that
- mlevental 7y ago
  https://media.giphy.com/media/gBpY4p7bbhsiI/giphy.gif https://media.giphy.com/media/gBpY4p7bbhsiI/giphy.gif

et2o 7y ago

childish troll attempt

halfarmbandit 7y ago

That’s quite naive... I am sure you could write stuff like Diffentiable Processors or the like from scratch with numpy but if you respect yourself and your time, you won’t. Complicated architectures are orders of magnitude harder than writing feed forward networks from scratch. For example, see the Merlin paper.