2 ms·

Very cool. This is somewhat naive question considering I actually have an EE background, and I think I know the answer but considering their shared EM theory, d

Very cool. This is somewhat naive question considering I actually have an EE background, and I think I know the answer but considering their shared EM theory, do you see any parallels of this thinking tangentially applicable to radio frequency system design?

fouronnes3 2y ago
I know absolutely nothing about radio so I can't really answer, sorry! But there's really something to be said about using PyTorch (or any other ML framework for that matter) as a general purpose optimizer. The modeling capabilities of torch.nn are quite extraordinary, and the fully dynamic nature of the PyTorch graph (something that wasn't really possible with previous frameworks like tensorflow) is really something that hasn't been talked about enough in my opinion. It's like differentiable programming, basically. You can write any "normal" python function and get an *exact* derivative of it. There are some caveats but it's very very powerful.
- MITSardine 2y ago
  Could you clarify this a little? My layman perception of NNs is they are a formalism that defines a parametric function family (for instance the family of affine functions has m + mxn parameters for n input and m output space dimensions) using some base primitives and composition rules. By tacking on a cost function, an optimizer, and a bunch of (input,output) pairs (training set) to this, one obtains optimal parameters such that the cost function is minimized over the training set (in some norm, I imagine). The NN can then be used to map never seen before inputs to outputs in a manner, hopefully, that leads to a small value of the loss function (i.e. adequately). Even if this is wrong to some extent, could you confirm that the optimizer is but one component of a larger system, and in fact one that exists independently of NNs as well? Such as a stochastic gradient descent. In that case, what is the role of NNs in what you mentioned, would it not be simpler to yank the optimizer out and apply it to your application directly? It seems to me trying to recast a given problem to a NN to make use of a Python library's included optimizer is a sort of "XY problem", if one could just write their cost function and pass it to the optimizer directly (which presumably is no less open source than the library that includes it). I may be misinterpreting, because this is not my field, however interpolation or projection are things very familiar to me, so I may have a bias to interpret things to resemble this. In that case, I'd welcome corrections.

fouronnes3 2y ago

I know absolutely nothing about radio so I can't really answer, sorry! But there's really something to be said about using PyTorch (or any other ML framework for that matter) as a general purpose optimizer. The modeling capabilities of torch.nn are quite extraordinary, and the fully dynamic nature of the PyTorch graph (something that wasn't really possible with previous frameworks like tensorflow) is really something that hasn't been talked about enough in my opinion. It's like differentiable programming, basically. You can write any "normal" python function and get an *exact* derivative of it. There are some caveats but it's very very powerful.

MITSardine 2y ago
Could you clarify this a little? My layman perception of NNs is they are a formalism that defines a parametric function family (for instance the family of affine functions has m + mxn parameters for n input and m output space dimensions) using some base primitives and composition rules. By tacking on a cost function, an optimizer, and a bunch of (input,output) pairs (training set) to this, one obtains optimal parameters such that the cost function is minimized over the training set (in some norm, I imagine). The NN can then be used to map never seen before inputs to outputs in a manner, hopefully, that leads to a small value of the loss function (i.e. adequately). Even if this is wrong to some extent, could you confirm that the optimizer is but one component of a larger system, and in fact one that exists independently of NNs as well? Such as a stochastic gradient descent. In that case, what is the role of NNs in what you mentioned, would it not be simpler to yank the optimizer out and apply it to your application directly? It seems to me trying to recast a given problem to a NN to make use of a Python library's included optimizer is a sort of "XY problem", if one could just write their cost function and pass it to the optimizer directly (which presumably is no less open source than the library that includes it). I may be misinterpreting, because this is not my field, however interpolation or projection are things very familiar to me, so I may have a bias to interpret things to resemble this. In that case, I'd welcome corrections.