5 ms·

In my current job, I write production machine learning code often in Scala. I had never written a line of Scala before that, had no idea what the idioms were no

by mlthoughts2018 8y ago

In my current job, I write production machine learning code often in Scala. I had never written a line of Scala before that, had no idea what the idioms were nor even what some of the inheritance and object privacy syntax meant. I read through mountains of legacy Scala code while working on my first several projects, and have assisted dozens of other new developers (most of whom also have never once read or thought about Scala previously). The assumptions people made in that codebase still give me nightmares, mostly over how foolish and lazy they are in little ways, little unexplained “self documented” things that create multi-hour stumbling blocks but which could have been written in a more “elementary” style in the first place (without sacrificing any necessary language features). When I see things like, “any C programmer should know ...” I just slap my head in stupor. It just misses the point entirely. It’s of zero importance whatsoever to write code in language X in a style that benefits mostly only “people who know language X.” Pretend you’re writing the code for a freshman intern who will look at it in 5 or 10 years with no prior knowledge of the language it’s written in, and now you’re in the correct ballpark for how to write professional software.

pantalaimon 8y ago
C does not have that many concepts and they are usually pretty straightforward. Is it too much to ask for someone to look them up if they find one they are not familiar with?
- mlthoughts2018 8y ago
  That seems like a completely backwards attitude. If you set out to write code to be maintainable and easily readable, you won’t think this way. Instead you’ll think, if I can write the code just slightly differently, slightly outside my own comfort zone or with less assumption about what burden should be on the reader, and it makes things much clearer for virtually no cost, why wouldn’t I? There’s no cost to me to write a separate assignment statement “i = i + 1” in many cases when I might otherwise write “i++”. The idea that it’s ok to put extraneous readership burden on the reader to nest “i++” in the middle of some other operator / indexing syntax / control flow logic is incompatible with writing good code. It has to start with prioritizing elementary clarity pedantically in cases when there’s no objective cost to disfavor it.
  - matfil 8y ago
    I'm going to argue that there is an objective cost, and that is the length of the code. To me, there's a massive win in being able to take in the whole of an algorithm on a single screen or sheet of paper, and using two statements to do something that can be idiomatically expressed in one is in direct opposition to that. I get the feeling that there are (at least) two camps here that can very easily talk past one another. In my experience, "succinct-code" advocates care about readability just as much as "avoid-language-specific-constructs" advocates. They just have different experiences of what maximally-readable code looks like.
    - mlthoughts2018 8y ago
      I agree with you there are multiple interpretations, but there’s an important distinction. For me personally or likely for other experienced engineers, code compactness is a nice thing. But this is unimportant, because nobody should be writing the code for me or with me in mind. Code compactness for the sake of other experienced engineers is an extremely stupid thing to prioritize above the simplicity of reading straightforward, separated operations that rely on as few concepts as possible. Code compactness is nowhere near as important as concept compactness, because the latter appeals to all engineers, not merely to other experienced ones. Anything that requires or introduces a new concept or mixes multiple concepts together is more costly in terms of hurting readability and maintainability than corresponding “longer” code that relies on fewer concepts.
      - matfil 8y ago
        I think I see where you're coming from, although I find it rather dispiriting, in that this point of view seems to be arguing against exploring and understanding the universe of different programming techniques. It would perhaps be instructive if you could enumerate or point to what you consider to be a good set of concepts to work from. Clearly there are some absurd answers ("it's all NAND gates in the end"), but I don't imagine that's what you're thinking. Would you include functors? applicative functors? Different forms of memory management? APL/R/Numpy-style arrays? setjmp/longjmp?
macintux 8y ago
Scala is a kitchen sink language, much like C++, one of the reasons I’m unfond of both. The number of things a programmer must know about C is much more constrained, and i++ is absolutely on that list.
- mlthoughts2018 8y ago
  I don’t see how the specifics of Scala are related to this discussion. The same thing happened to me in C in fact back in the 00s when I was doing scientific computing for a government research lab that had a large and critical collection of C programs written between the late 80s and late 90s. It happens to all programming languages when engineers don’t prioritize future-proof readability as a first class part of maintainability along with testing and judicious choices about when to invest in extensibility designs vs when to ignore extensibility.
  - userbinator 8y ago
    when engineers don’t prioritize future-proof readability What does it say about the competence of "future" engineers if they can't understand code written by past ones? One of my favourite sayings is "the code is unreadable to you, because you are not qualified to understand it yet". Perhaps we should not encourage the degradation of our craft.
    - mlthoughts2018 8y ago
      > “What does it say about the competence of "future" engineers if they can't understand code written by past ones?” It doesn’t say much about those future engineers at all. It’s easy to write inscrutable code that even veteran engineers can’t understand, yet still “gets the job done.” Also, you act like this would imply that all future engineers are the same, but that is not true and not related to my points. You’re writing code for the inexperienced future engineer, some poor soul tossed into a legacy codebase without much help. It’s not their fault they were put in that position. It’s sink or swim. Good code helps _that_ person swim. As for the quote you mention, > “One of my favourite sayings is "the code is unreadable to you, because you are not qualified to understand it yet". That is a horrible way of thinking, with built in condescending attitudes and everything. If code is unreadable to someone, beyond basic syntax definitions, that is the author’s fault and not at all the reader’s fault for being inexperienced. The reason 5th graders can’t read Infinite Jest is because David Foster Wallace tried to make it complicated, not because, in some skewed perspective, the writing is perfectly simple but 5th graders just aren’t experienced enough yet. It’s a complexity property of the writing not of the reader’s brain. With literature you can get away with this because there are extenuating arguments about artistic merit. For business software, not so. In that case, if you write something more like DFW and less like Hemingway, it’s a sign of laziness and lack of self-discipline.. and not at all a sign of skill advancement.
armitron 8y ago
Software Engineering must be the only engineering discipline where mastery of tools and barrier to entry in terms of skills are in many ways inversely correlated with market demands. Every clown can write software these days and in fact a lot of clowns get paid six figure salaries to do so. When the domain is largely composed of clowns, standards become extinct and one finds oneself calibrating for a circus. If / when software liabilities become real, this fiesta of the clueless will come to a rapid stop. Let us hope that the damage done by that point will not be too great.
- mlthoughts2018 8y ago
  I actually find comments like this to be some mix of funny & infuriating because it is so dramatically false. There are all these myths about “one bad hire” or how a bad engineer can ruin a team, but they are mostly overblown bravado by coder cowboys who think they know how to do everything right and should just be left to code in a vacuum, and that anyone who dares challenge them or demand compromise with real world limitations, deadlines, customer demands, must be a “clown” (as you put it). It’s a sad state of arrogance and I think it leads directly to the joke we have today for software interviews, code tests, TripleByte / CoderPad garbage using language esoterica and whiteboard trivia to haze out anyone with common sense about business software. Most code that creates value to people is messy, written in a hurry with vague specifications & unclear understanding of what the end user wants or would pay for, and it gets iterated by disjoint teams of people with competing timelines, politics, credit mongering and resource constraints, and most of it is for reporting something to someone. If you don’t step back and realize that bare metal performance and esoteric data structures and algorithms generally don’t matter and can be picked up quickly by just about any engineer, and that real value comes from putting careful scaffolding and comments around hacked up business logic that needs to be future proofed against highly volatile business circumstances, you’re just going to be miserable and probably should quit and try to find a financially viable open source job on a compiler team or something where the exceedingly rare assumption you want to enforce (that most people reading your code are above average at the idioms of the language or tool you’re using) has some prayer of being relevant or applicable.
  - 8y ago
userbinator 8y ago
It’s of zero importance whatsoever to write code in language X in a style that benefits mostly only “people who know language X.” with no prior knowledge of the language it’s written in W...T...F...? Someone who can't understand the code they've been assigned to work on is simply unqualified for the job.
- mlthoughts2018 8y ago
  > “Someone who can't understand the code they've been assigned to work on” Nobody said anything like this! I don’t understand how you make this conflation mistake. Before I ever read one single character of Scala code, I had written hundreds of thousands of lines of code in C, Python and Haskell, among others. It was easy to understand Scala code despite having never read it except for places where people tried to rely on needless shorthand or obtuse ways of doing things. I can’t understand your response. It’s so easy to write C code that a veteran C expert has to spend hours to understand, or to rewrite the same code in a way where a freshman who only ever used Java could equally easily understand it. Familiarity with how to unlock meaning from badly written code is not the same thing as experience with a language.
gamma-male 8y ago
The problem is that a lot of C devs have been coding in C and only in C all their lives. They cannot accept that people coming from other languages want to be able to read their code, or that a security audit needs to be done over thousands and thousands of their lines and that these C hacks are an awful thing to read for someone who wants to quickly review the code.
- mlthoughts2018 8y ago
  Yes, I agree with you. That is the driving force, even with other languages besides C too, and influences many things, from code readability to interview techniques.
- userbinator 8y ago
  Replace C with Japanese or Chinese and you'll see just how absurd your argument is. If you want to read something in a language you don't know, you learn the language. Every profession has its own terms of art, its own abbreviations and lingo, and to suggest that those don't have any value is just ridiculous. I seriously don't understand this attitude. You don't see other professions saying how they should be understandable by everyone who hasn't studied them. Why should programming be any different? I blame the "everyone can code" charlatans...
  - mlthoughts2018 8y ago
    It seems like your analogy with other natural languages is just a false comparison. You say, > “If you want to read something in a language you don't know, you learn the language” but this means you’re missing the point. If you are writing something like a tech manual in Japanese, and you know ahead of time that it will be required that sometimes people totally unfamiliar with nuanced Japanese (but with high skill at picking up badic ideas in any language) have to rely on your manual, then how should you write it? The foolish answer is to say you’ll write it with advanced Japanese language constructs and then turn around and say, “oh well, novices who want to rely on the manual should have gone and learned advanced Japanese.” The smart answer is to say you’ll use self-discipline and restrain yourself from nuanced Japanese, and instead write in a way that novices have a good chance of decoding with minimal extra effort, and of course advanced Japanese readers will also still be able to get what they need too. This is such a self-centered idea, approaching it like, “I can write whatever code I want using whatever idioms and it’s someone else’s fault if they didn’t learn the language sufficiently ahead of time before they found themselves needing to quickly read my code.”

pantalaimon 8y ago

C does not have that many concepts and they are usually pretty straightforward. Is it too much to ask for someone to look them up if they find one they are not familiar with?

mlthoughts2018 8y ago
That seems like a completely backwards attitude. If you set out to write code to be maintainable and easily readable, you won’t think this way. Instead you’ll think, if I can write the code just slightly differently, slightly outside my own comfort zone or with less assumption about what burden should be on the reader, and it makes things much clearer for virtually no cost, why wouldn’t I? There’s no cost to me to write a separate assignment statement “i = i + 1” in many cases when I might otherwise write “i++”. The idea that it’s ok to put extraneous readership burden on the reader to nest “i++” in the middle of some other operator / indexing syntax / control flow logic is incompatible with writing good code. It has to start with prioritizing elementary clarity pedantically in cases when there’s no objective cost to disfavor it.
- matfil 8y ago
  I'm going to argue that there is an objective cost, and that is the length of the code. To me, there's a massive win in being able to take in the whole of an algorithm on a single screen or sheet of paper, and using two statements to do something that can be idiomatically expressed in one is in direct opposition to that. I get the feeling that there are (at least) two camps here that can very easily talk past one another. In my experience, "succinct-code" advocates care about readability just as much as "avoid-language-specific-constructs" advocates. They just have different experiences of what maximally-readable code looks like.
  - mlthoughts2018 8y ago
    I agree with you there are multiple interpretations, but there’s an important distinction. For me personally or likely for other experienced engineers, code compactness is a nice thing. But this is unimportant, because nobody should be writing the code for me or with me in mind. Code compactness for the sake of other experienced engineers is an extremely stupid thing to prioritize above the simplicity of reading straightforward, separated operations that rely on as few concepts as possible. Code compactness is nowhere near as important as concept compactness, because the latter appeals to all engineers, not merely to other experienced ones. Anything that requires or introduces a new concept or mixes multiple concepts together is more costly in terms of hurting readability and maintainability than corresponding “longer” code that relies on fewer concepts.
    - matfil 8y ago
      I think I see where you're coming from, although I find it rather dispiriting, in that this point of view seems to be arguing against exploring and understanding the universe of different programming techniques. It would perhaps be instructive if you could enumerate or point to what you consider to be a good set of concepts to work from. Clearly there are some absurd answers ("it's all NAND gates in the end"), but I don't imagine that's what you're thinking. Would you include functors? applicative functors? Different forms of memory management? APL/R/Numpy-style arrays? setjmp/longjmp?

macintux 8y ago

Scala is a kitchen sink language, much like C++, one of the reasons I’m unfond of both. The number of things a programmer must know about C is much more constrained, and i++ is absolutely on that list.

mlthoughts2018 8y ago
I don’t see how the specifics of Scala are related to this discussion. The same thing happened to me in C in fact back in the 00s when I was doing scientific computing for a government research lab that had a large and critical collection of C programs written between the late 80s and late 90s. It happens to all programming languages when engineers don’t prioritize future-proof readability as a first class part of maintainability along with testing and judicious choices about when to invest in extensibility designs vs when to ignore extensibility.
- userbinator 8y ago
  when engineers don’t prioritize future-proof readability What does it say about the competence of "future" engineers if they can't understand code written by past ones? One of my favourite sayings is "the code is unreadable to you, because you are not qualified to understand it yet". Perhaps we should not encourage the degradation of our craft.
  - mlthoughts2018 8y ago
    > “What does it say about the competence of "future" engineers if they can't understand code written by past ones?” It doesn’t say much about those future engineers at all. It’s easy to write inscrutable code that even veteran engineers can’t understand, yet still “gets the job done.” Also, you act like this would imply that all future engineers are the same, but that is not true and not related to my points. You’re writing code for the inexperienced future engineer, some poor soul tossed into a legacy codebase without much help. It’s not their fault they were put in that position. It’s sink or swim. Good code helps _that_ person swim. As for the quote you mention, > “One of my favourite sayings is "the code is unreadable to you, because you are not qualified to understand it yet". That is a horrible way of thinking, with built in condescending attitudes and everything. If code is unreadable to someone, beyond basic syntax definitions, that is the author’s fault and not at all the reader’s fault for being inexperienced. The reason 5th graders can’t read Infinite Jest is because David Foster Wallace tried to make it complicated, not because, in some skewed perspective, the writing is perfectly simple but 5th graders just aren’t experienced enough yet. It’s a complexity property of the writing not of the reader’s brain. With literature you can get away with this because there are extenuating arguments about artistic merit. For business software, not so. In that case, if you write something more like DFW and less like Hemingway, it’s a sign of laziness and lack of self-discipline.. and not at all a sign of skill advancement.

armitron 8y ago

Software Engineering must be the only engineering discipline where mastery of tools and barrier to entry in terms of skills are in many ways inversely correlated with market demands. Every clown can write software these days and in fact a lot of clowns get paid six figure salaries to do so. When the domain is largely composed of clowns, standards become extinct and one finds oneself calibrating for a circus. If / when software liabilities become real, this fiesta of the clueless will come to a rapid stop. Let us hope that the damage done by that point will not be too great.

mlthoughts2018 8y ago
I actually find comments like this to be some mix of funny & infuriating because it is so dramatically false. There are all these myths about “one bad hire” or how a bad engineer can ruin a team, but they are mostly overblown bravado by coder cowboys who think they know how to do everything right and should just be left to code in a vacuum, and that anyone who dares challenge them or demand compromise with real world limitations, deadlines, customer demands, must be a “clown” (as you put it). It’s a sad state of arrogance and I think it leads directly to the joke we have today for software interviews, code tests, TripleByte / CoderPad garbage using language esoterica and whiteboard trivia to haze out anyone with common sense about business software. Most code that creates value to people is messy, written in a hurry with vague specifications & unclear understanding of what the end user wants or would pay for, and it gets iterated by disjoint teams of people with competing timelines, politics, credit mongering and resource constraints, and most of it is for reporting something to someone. If you don’t step back and realize that bare metal performance and esoteric data structures and algorithms generally don’t matter and can be picked up quickly by just about any engineer, and that real value comes from putting careful scaffolding and comments around hacked up business logic that needs to be future proofed against highly volatile business circumstances, you’re just going to be miserable and probably should quit and try to find a financially viable open source job on a compiler team or something where the exceedingly rare assumption you want to enforce (that most people reading your code are above average at the idioms of the language or tool you’re using) has some prayer of being relevant or applicable.
- 8y ago

userbinator 8y ago

It’s of zero importance whatsoever to write code in language X in a style that benefits mostly only “people who know language X.” with no prior knowledge of the language it’s written in W...T...F...? Someone who can't understand the code they've been assigned to work on is simply unqualified for the job.

mlthoughts2018 8y ago
> “Someone who can't understand the code they've been assigned to work on” Nobody said anything like this! I don’t understand how you make this conflation mistake. Before I ever read one single character of Scala code, I had written hundreds of thousands of lines of code in C, Python and Haskell, among others. It was easy to understand Scala code despite having never read it except for places where people tried to rely on needless shorthand or obtuse ways of doing things. I can’t understand your response. It’s so easy to write C code that a veteran C expert has to spend hours to understand, or to rewrite the same code in a way where a freshman who only ever used Java could equally easily understand it. Familiarity with how to unlock meaning from badly written code is not the same thing as experience with a language.

gamma-male 8y ago

The problem is that a lot of C devs have been coding in C and only in C all their lives. They cannot accept that people coming from other languages want to be able to read their code, or that a security audit needs to be done over thousands and thousands of their lines and that these C hacks are an awful thing to read for someone who wants to quickly review the code.

mlthoughts2018 8y ago
Yes, I agree with you. That is the driving force, even with other languages besides C too, and influences many things, from code readability to interview techniques.
userbinator 8y ago
Replace C with Japanese or Chinese and you'll see just how absurd your argument is. If you want to read something in a language you don't know, you learn the language. Every profession has its own terms of art, its own abbreviations and lingo, and to suggest that those don't have any value is just ridiculous. I seriously don't understand this attitude. You don't see other professions saying how they should be understandable by everyone who hasn't studied them. Why should programming be any different? I blame the "everyone can code" charlatans...
- mlthoughts2018 8y ago
  It seems like your analogy with other natural languages is just a false comparison. You say, > “If you want to read something in a language you don't know, you learn the language” but this means you’re missing the point. If you are writing something like a tech manual in Japanese, and you know ahead of time that it will be required that sometimes people totally unfamiliar with nuanced Japanese (but with high skill at picking up badic ideas in any language) have to rely on your manual, then how should you write it? The foolish answer is to say you’ll write it with advanced Japanese language constructs and then turn around and say, “oh well, novices who want to rely on the manual should have gone and learned advanced Japanese.” The smart answer is to say you’ll use self-discipline and restrain yourself from nuanced Japanese, and instead write in a way that novices have a good chance of decoding with minimal extra effort, and of course advanced Japanese readers will also still be able to get what they need too. This is such a self-centered idea, approaching it like, “I can write whatever code I want using whatever idioms and it’s someone else’s fault if they didn’t learn the language sufficiently ahead of time before they found themselves needing to quickly read my code.”