5 ms·

Why are deep learning researchers allergic to meaningful variable names?

chrisfosterelli 8y ago
These are meaningful variable names in the context of a code implementation of a math algorithm. The variables directly map to the algorithmic notation.
- jacobolus 8y ago
  And when trying to do algebraic manipulation on paper, having an 8 or 10 letter variable name is incredibly cumbersome. Frankly in the middle of a numerical algorithm it is typically also cumbersome in code to have descriptive variable names for everything. However, mathematical code (especially when written by scientists, etc.) often takes this too far, introducing many 1- or few-letter variable names without enough context or description to figure out what they stand for, and for more of the variables than necessary. This blog post was just fine though. h = hidden, o = output, y_pred = predicted value of y, etc. are quite clear
  - gambler 8y ago
    >h = hidden, o = output, y_pred = predicted value of y, etc. are quite clear ∀ abbr + to the cogntv load prsn wht mntn in thr hd. ∀ -> every + -> adds wht -> who has to h -> head Was that easier to read than the following? Every abbreviation adds to the cognitive load the person has to maintain in their head. This isn't code, but the principles of making code (un)readable are exactly the same. If you see that full-word variable names create too much clutter, it means your code structure is wrong. So you fix it. You go from this: self.w1 = np.random.normal() self.w2 = np.random.normal() self.w3 = np.random.normal() self.w4 = np.random.normal() self.w5 = np.random.normal() self.w6 = np.random.normal() To this: set_to_random_numbers(self.weights) After 9 years of maintaining several dozen legacy codebases (which sometimes involved financial math) I am fully convinced that the only people who like terse code are people who write it and never read it after.
    - jacobolus 8y ago
      Are you suggesting that we should write all of our mathematical expressions using prose, and stop using symbols for operators? That would be the original approach historically (before the past 500 years), e.g. for the quadratic formula, Brahmagupta (628 CE): > To the absolute number multiplied by four times the square, add the square of the middle term; the square root of the same, less the middle term, being divided by twice the square is the value. Go ahead and do what you like, but I doubt you’ll find many publishers who will accept your paper in the 21st century. I know which one imposes more cognitive load for me. But disclaimer: I spent a lot of time from age 5–20 working with mathematical notation.
      - deleted 8y ago
        [deleted]
      - gambler 8y ago
        I am suggesting that just because software does "math" doesn't change how people read it. Bad coding practices like cryptic variable names and repetitions will affect it exactly in the same ways they affect software in all other domains.
        ska 8y ago
        I see what you are saying, but context matters. In code that starts something like "/* implements gambler et. al 2019 (doi:xxxxxxx) eqn 3 ... */". Then I really expect the code to go to great lengths to match the notation used in the paper. Anything else is adding to the cognitive load. The exception is if the entire algorithm is discussed in the comments of the code without outside reference, then I want the code and comment to be extremely consistent. Personally, I like the former as a shorthand for "don't mess with this without the paper in front of you, you'll probably screw it up".
        jacobolus 8y ago
        > just because software does "math" doesn't change how people read it. It absolutely does. Different problem domains (and different communities’ treatment of problems) involve differing types and amounts of formal structure, differing conventional notations, etc., and in practice the code looks substantially different (in organization, abstractions used, naming, ...) even if you try to standardize it to all look the same. People who are reading “math” code can be expected to understand mathematical notation, e.g. to be capable of reading a journal paper where an algorithm or formula is described more completely including motivation, formal derivation, proofs of correctness, proofs of various formal properties, ... Mathematical code is often quite abstract; unlike company-specific business logic, the same tool might be used in many different contexts with inputs of different meanings. There really isn’t that much insight gained by replacing i with “index”, x with “generic_input_variable”, or to take it to an extreme, ax + b with add(multiply(input_variable, proportionality_constant), offset_constant) or sin(x) with perpendicular_unit_vector_component_for_angle(input_angle_measure) The extra space overhead of the long variables and words instead of symbols is a killer for clarity. If variable names are “cryptic” [as in, can’t be guessed at a glance by someone working in the field] then that is indeed a failure though. Short variable names should have limited scope (ideally fitting on one screenfull of code) and obvious meaning in context, which might involve some explanatory comments, links to a paper.
thatoneuser 8y ago
They're typically not coders as a background. They're usually a hard science/math person so they didn't get variable indoctrination. I haven't coded a NN since grad school and just scanning over the code I can tell this is basically the same as what I did. No doubt this is a boiler plate approach compiled into an article.
x220 8y ago
Because you'll look more impressive if fewer people quickly understand your code.
fny 8y ago
I'm not sure what you mean in this case: the Python code is very well written and extremely well commented. I think this is an extremely well written tutorial! Kudos to the author.
_raoulcousins 8y ago
I've been told that not knowing better is a typical reason (and another reply says so), but my anecdotal contribution is different. When I'm implementing algorithm, I'll typically be implementing something that uses single letter variable names. It's easier to do with consistent variable names. That's it. I once tried meaningful variable names in LaTeX to make code nicer, but long variable names rendered as math are an unreadable mess, so I went back to short variable names. That being said, it's not hard to change variable names with a good IDE, and in LaTeX you can alias variable names with \newcommand.
gmiller123456 8y ago
They're usually working from text/notes in mathematical notation and are trying to match the math notation as best they can. I'm sure programming not being their profession may be part of it, but I find myself doing the same thing when implementing algorithms described in mathematical notation. Due to the shear number of variables in a lot of algorithms, it's difficult to give each one a good name. And even if you did, it's unlikely the code would make any more sense, in order to grasp what's going on, you'd likely need some text on the subject to explain how it works.
manfredo 8y ago
I've found that lots of people from math backgrounds are comfortable with non-descriptive variable names. Most variables in math are Greek letters, or "i, j, k" and things like that.
nilkn 8y ago
For what it's worth, I didn't read the article at all, went straight to the final code, and had no issue reading or understanding it. The variable names seem fine to me. They're actually quite good, and I liked the convention for naming partial derivatives. This code is designed to be read by someone who is pretty familiar with the underlying concepts. If you don't really know how a neural network works or how gradient descent is applied to update the weights, that's your blocker, not the variable names I saw here.
- deytempo 8y ago
  Well then they should get rid of the misleading title “from scratch” it makes it sound like they are going to explain it without beltching out a bunch of abstract math. I’m really annoyed that everyone pretends that you need math to explain a NN. I have done plenty of them without numpy or math using just arrays and loops.
  - nilkn 8y ago
    This is not abstract math, and the article does explain what it's doing before presenting the code snippets. How can you explain or implement gradient descent without math? At some point I think you have to accept that this is a topic that involves math, and you're way better off understanding it on those terms rather than trying to avoid it.

These are meaningful variable names in the context of a code implementation of a math algorithm. The variables directly map to the algorithmic notation.

jacobolus 8y ago
And when trying to do algebraic manipulation on paper, having an 8 or 10 letter variable name is incredibly cumbersome. Frankly in the middle of a numerical algorithm it is typically also cumbersome in code to have descriptive variable names for everything. However, mathematical code (especially when written by scientists, etc.) often takes this too far, introducing many 1- or few-letter variable names without enough context or description to figure out what they stand for, and for more of the variables than necessary. This blog post was just fine though. h = hidden, o = output, y_pred = predicted value of y, etc. are quite clear
- gambler 8y ago
  >h = hidden, o = output, y_pred = predicted value of y, etc. are quite clear ∀ abbr + to the cogntv load prsn wht mntn in thr hd. ∀ -> every + -> adds wht -> who has to h -> head Was that easier to read than the following? Every abbreviation adds to the cognitive load the person has to maintain in their head. This isn't code, but the principles of making code (un)readable are exactly the same. If you see that full-word variable names create too much clutter, it means your code structure is wrong. So you fix it. You go from this: self.w1 = np.random.normal() self.w2 = np.random.normal() self.w3 = np.random.normal() self.w4 = np.random.normal() self.w5 = np.random.normal() self.w6 = np.random.normal() To this: set_to_random_numbers(self.weights) After 9 years of maintaining several dozen legacy codebases (which sometimes involved financial math) I am fully convinced that the only people who like terse code are people who write it and never read it after.
  - jacobolus 8y ago
    Are you suggesting that we should write all of our mathematical expressions using prose, and stop using symbols for operators? That would be the original approach historically (before the past 500 years), e.g. for the quadratic formula, Brahmagupta (628 CE): > To the absolute number multiplied by four times the square, add the square of the middle term; the square root of the same, less the middle term, being divided by twice the square is the value. Go ahead and do what you like, but I doubt you’ll find many publishers who will accept your paper in the 21st century. I know which one imposes more cognitive load for me. But disclaimer: I spent a lot of time from age 5–20 working with mathematical notation.
    - deleted 8y ago
      [deleted]
    - gambler 8y ago
      I am suggesting that just because software does "math" doesn't change how people read it. Bad coding practices like cryptic variable names and repetitions will affect it exactly in the same ways they affect software in all other domains.
      - ska 8y ago
        I see what you are saying, but context matters. In code that starts something like "/* implements gambler et. al 2019 (doi:xxxxxxx) eqn 3 ... */". Then I really expect the code to go to great lengths to match the notation used in the paper. Anything else is adding to the cognitive load. The exception is if the entire algorithm is discussed in the comments of the code without outside reference, then I want the code and comment to be extremely consistent. Personally, I like the former as a shorthand for "don't mess with this without the paper in front of you, you'll probably screw it up".
      - jacobolus 8y ago
        > just because software does "math" doesn't change how people read it. It absolutely does. Different problem domains (and different communities’ treatment of problems) involve differing types and amounts of formal structure, differing conventional notations, etc., and in practice the code looks substantially different (in organization, abstractions used, naming, ...) even if you try to standardize it to all look the same. People who are reading “math” code can be expected to understand mathematical notation, e.g. to be capable of reading a journal paper where an algorithm or formula is described more completely including motivation, formal derivation, proofs of correctness, proofs of various formal properties, ... Mathematical code is often quite abstract; unlike company-specific business logic, the same tool might be used in many different contexts with inputs of different meanings. There really isn’t that much insight gained by replacing i with “index”, x with “generic_input_variable”, or to take it to an extreme, ax + b with add(multiply(input_variable, proportionality_constant), offset_constant) or sin(x) with perpendicular_unit_vector_component_for_angle(input_angle_measure) The extra space overhead of the long variables and words instead of symbols is a killer for clarity. If variable names are “cryptic” [as in, can’t be guessed at a glance by someone working in the field] then that is indeed a failure though. Short variable names should have limited scope (ideally fitting on one screenfull of code) and obvious meaning in context, which might involve some explanatory comments, links to a paper.

thatoneuser 8y ago

They're typically not coders as a background. They're usually a hard science/math person so they didn't get variable indoctrination. I haven't coded a NN since grad school and just scanning over the code I can tell this is basically the same as what I did. No doubt this is a boiler plate approach compiled into an article.

x220 8y ago

Because you'll look more impressive if fewer people quickly understand your code.

fny 8y ago

I'm not sure what you mean in this case: the Python code is very well written and extremely well commented. I think this is an extremely well written tutorial! Kudos to the author.

_raoulcousins 8y ago

I've been told that not knowing better is a typical reason (and another reply says so), but my anecdotal contribution is different. When I'm implementing algorithm, I'll typically be implementing something that uses single letter variable names. It's easier to do with consistent variable names. That's it. I once tried meaningful variable names in LaTeX to make code nicer, but long variable names rendered as math are an unreadable mess, so I went back to short variable names. That being said, it's not hard to change variable names with a good IDE, and in LaTeX you can alias variable names with \newcommand.

gmiller123456 8y ago

They're usually working from text/notes in mathematical notation and are trying to match the math notation as best they can. I'm sure programming not being their profession may be part of it, but I find myself doing the same thing when implementing algorithms described in mathematical notation. Due to the shear number of variables in a lot of algorithms, it's difficult to give each one a good name. And even if you did, it's unlikely the code would make any more sense, in order to grasp what's going on, you'd likely need some text on the subject to explain how it works.

manfredo 8y ago

I've found that lots of people from math backgrounds are comfortable with non-descriptive variable names. Most variables in math are Greek letters, or "i, j, k" and things like that.

nilkn 8y ago

For what it's worth, I didn't read the article at all, went straight to the final code, and had no issue reading or understanding it. The variable names seem fine to me. They're actually quite good, and I liked the convention for naming partial derivatives. This code is designed to be read by someone who is pretty familiar with the underlying concepts. If you don't really know how a neural network works or how gradient descent is applied to update the weights, that's your blocker, not the variable names I saw here.

deytempo 8y ago
Well then they should get rid of the misleading title “from scratch” it makes it sound like they are going to explain it without beltching out a bunch of abstract math. I’m really annoyed that everyone pretends that you need math to explain a NN. I have done plenty of them without numpy or math using just arrays and loops.
- nilkn 8y ago
  This is not abstract math, and the article does explain what it's doing before presenting the code snippets. How can you explain or implement gradient descent without math? At some point I think you have to accept that this is a topic that involves math, and you're way better off understanding it on those terms rather than trying to avoid it.