3 ms·
This looks amazing and if works as advertised will be a huge productivity breakthrough. The number of times I've sat with 2 developers looking over my shoulder
by nl 6y ago
This looks amazing and if works as advertised will be a huge productivity breakthrough.
The number of times I've sat with 2 developers looking over my shoulder trying to workout WTF my TF (or Numpy or Pytorch) manipulations aren't ending up the correct shape is huge. It's probably $50-100K/year of time for the last 4-5 years.
It's always obvious in retrospect.. until next time you have to read that code.
- raverbashing 6y agoThat is so true. Keras tries to help you, but the messages are inscrutable sometimes. "Oh there's this extra dimension that has no size, and this other that's just size 1 but if you try to flatten it everything stops working"
- mcabbott 6y ago> until next time you have to read that But how will this tool help? If you put the sample you used in a comment, then your confusion is wondering if that's in sync... and whether the input is like what you wrote. IMO the sweet spot that's easy both for humans & computers is index notation. Instead of `tf.add(X, tf.expand_dims(Y, 1))` (their example, which I think also applies to 4-tensors in ways not trivial to see from the sample) something like "out_{r,c} = X_r + Y_c" is completely unambiguous, and quicker to type than making a sample. I don't think you can do that in tf.einsum, nor einops. Is there a Python package which does this? In Julia you can write `@cast out[r,c] := X[r] + Y[c]` (with my package, but several others share this notation).
- nl 6y ago>> until next time you have to read that > But how will this tool help? It's absolutely true that it doesn't help. But that's a second order problem to the hours spend trying to work out how to write the code. It could also generate comments that explain what is happening..
- im3w1l 6y agoI recommend X + Y[:, tf.newaxis] For higher dimensions of Y, you can use (but it might be ill-advised) X + Y[:, tf.newaxis, ...]
- mcabbott 6y agoSure! The desire for fancier notation is from more complicated examples, the kind where you write elaborate comments to explain why you're permuting 3rd & 4th dim of B to line up with C and part of D.
- SQueeeeeL 6y agoI mean, advanced tools will just create people with even less understanding of what's going on. And thus have even less of an idea how to fix something when "Torch is being dumb" (no it's not Tom! You just suck, stop ripping templates off stackoverflow and actually read some documentation and understand what functions you're using do!) 99pi has a great episode on this phenomenon related to autopilot in planes https://99percentinvisible.org/episode/children-of-the-magenta-automation-paradox-pt-1/ https://99percentinvisible.org/episode/children-of-the-magen... where over automation makes edge case situations much harder to handle (in the case of planes this is deadly)
- nl 6y agoIt's often (for me anyway) not the understanding, it's the syntax that is a problem. I don't see any problem in automating the generation of the line-noise that is array manipulations. Can I write assembly? No. Do I understand what a compiler is doing? Yes. Does this cause problems for me? Maybe twice in my 20 year career and in both cases I've worked it out. Seems like a good trade off, and I see this as similar.