3 ms·

You didn't ask about this, but a salient point to me: when programs were created on punched cards in the early 70's, and you had to wait 15 minutes to an hour

You didn't ask about this, but a salient point to me: when programs were created on punched cards in the early 70's, and you had to wait 15 minutes to an hour for a single compile, you spent a lot of time proofreading your source code before submitting it. You designed and coded the entire program in advance, none of this programming by accretion that is so common now. And we became quite skilled at finding our own errors before submitting code to the compiler. The current approach of letting the compiler lead you by the nose from one error to the next seems horribly inefficient to me, and results in a higher defect rate in the finished product.

afarrell 6y ago
It seems like that to me as well, but what is an engineer to do if he feels confused about what he's building on top of and he gets told that he's just feeling impostor syndrome or needs to stop "trying to understand the universe" or that his boss trusts that he does actually understand things? It has been true my whole career and I can't wait for the moment when I can retire and then have the time to learn enough to feel like I know what I'm doing.
- Delk 6y ago
  The "universe" is also, in some sense, much more complex than a couple of decades ago. In most cases we don't need to deal with as much low-level complexity or with cumbersome tooling, but "understanding the universe" would mean understanding pretty much all the levels of abstraction we have (and we have a lot), a diverse stack of technologies with rather different paradigms, a security landscape that has become much more complex with all the code we're running in a browser, and so on. I don't know what one is to do, but I somehow feel that "understanding the universe" is much more difficult nowadays than it perhaps was a few decades ago. There were probably fewer levels of abstraction, less diversity of technology, UIs might have been programmed in the same language as the rest of the program so there might not have been a need to juggle in a few languages at the same time, etc. Not to mention that since it's nowadays possible to program reasonably without having all that lower-level understanding, unlike perhaps a few decades ago, lots of people do that and (immediate) productivity expectations naturally grow to match. I feel the pain, though.
  - alisonatwork 6y ago
    I agree that there were less levels of abstraction, but I think the domain in which each individual developer operates isn't that much different. In my first "serious" programming job, I was hired to be a C programmer. But just to get C building, you also need to understand Makefiles, and to understand those you need to understand shell script. We printed stuff to console in color, so we needed to understand ANSI escape sequences. We also printed to printers, so we needed to understand HP PCL. Some of our systems needed to talk to dedicated hardware attached on the serial port, so we had to learn proprietary protocols. Newer devices were attached via TCP, so we had to learn sockets and those protocols too. We had our own home-built database, but sometimes we needed SQL. We used Informix, later Postgres. Each had a different stored procedure language and C bindings. We deployed on UNIX, so we needed to understand files and pipes and processes and semaphores. We used a modem to dial our our customer sites, so needed to know about a tool called Kermit. There were many different flavors of UNIX back then, and they all had slightly different behaviors. We had to learn all the quirks that applied to the APIs we were using. Later on we started to build front ends in Java AWT and Python GTK, and that brought in a slew of new tech. Much of this tech was still pre-2000. In my current job, I am mostly working in Java and Python, but now they are backend languages. The services are built using Gradle, which needs knowledge of Groovy, or setuptools which is just more Python. The packages are bundled in containers and pushed to the cloud. There is a lot of engineering that enables this stack, but the only part we need to know about is the format of a Dockerfile (so, basic shell scripting) and various YAMLs for CloudFormation and the CI/CD system. We talk to lots of other services, but we don't need to know anything about TCP or sockets, we just need to use a library to manage the HTTP connections and another library to convert the wire format to objects. We don't care about presentation logic or reporting because someone elsewhere in the stack is going to do that. We define our contracts in Protobuf, JSON or XML and move on. Some services need to keep state, and for almost all use cases the file system is sufficient - in AWS that means S3. Sometimes we send or receive messages - in AWS that means SNS/SQS. We also push metrics and logging and distributed trace context, but that's just more HTTP abstracted away into a library. We build dashboards in Grafana using OpenTSDB query language. We check our logs in Kibana using Lucene query language. There is a lot to learn. I could spend even more time learning about Kubernetes or Kafka or Go or Redis or a hundred other components required by software outside my current domain, but none of that is necessary in my current role. Back in the day I could have spent time learning about OS/390 or Syslog or Perl or Berkeley DB too. So, although I agree that "the universe" in general is much more complex, I haven't found that to be true about the part individual developers are expected to understand.
- alisonatwork 6y ago
  At a certain point you need to trust that the library you are calling or the framework you are developing inside is doing what it is supposed to do. Going deeper than that can be interesting as a learning experience, and occasionally it's required to figure out really difficult bugs or performance issues, but it's not a prerequisite for being able to understand your own work. Identify the edges of your domain, accept them and treat them as the rules of the game. That said, by the sounds of it you have worked with some colleagues who have been unhelpful in teaching you the rules. This might be because you are building on a poorly-architected thing that cannot easily be explained, or in a system that does not have detailed documentation. Both of those situations are solvable problems, and neither of them is an excuse for the people who do understand to gatekeep their knowledge or discourage curiosity.
gregjor 6y ago
This was called “desk checking.” You had to learn to read code and actually understand what it was doing. You may only get a few chances a day to run your code. I remember the feeling submitting my job (a deck of punch cards) and then getting my cards and a small printout back in a few minutes because of an ABEND (abnormal end, or crash). I still do that, along with targeted experimenting on small bits of code I need to understand better. The skill of reading code and “running” it in the head seems forgotten now, though I think talented programmers usually get there.
- alisonatwork 6y ago
  One of my biggest disappointments in pair programming over the past few years is realizing how many developers are unable to explain what the code they just wrote actually does. Like, they can't do a verbal debug, stepping through each line and describing the expected state. Everyone copies from the best, but it seems a lot of developers just copy, paste and then let the IDE autofix any warnings. The code block becomes a magical incantation rather than a sequence of operations. It never occurred to me that the reason I look at code differently is simply because I am from am older generation. I only started coding professionally in the 90s, but even then compiling wasn't especially fast so it was expected to review code before kicking off a build. And as a kid in the 80s I spent more time reading code in books and magazines than actually at a computer typing it in, so I had to think about and imagine what it would do. There is definitely a lot to be said for the modern world of linters, unit tests etc running continuously - I wouldn't want to give up any of that up! But I do wonder if the developers who aren't able to explain what their code actually does are missing some fundamental skill. Clearly it doesn't matter to the customer because these developers still produce working products, but it feels to me like some level of quality has been lost.
  - gregjor 6y ago
    My experience as well. Combined with the irresistible attraction to new and increasingly complex frameworks, tools, and infrastructure, this leads to a lot of failed projects. As an aging “not a good culture fit” programmer I found a niche taking on, finishing, and supporting the code left behind by developers who got in over their head and didn’t really understand what their code does.
quickthrower2 6y ago
I’d find trying to create a perfect program pre-compile and pre-unit test horribly inefficient. OTOH I think programmers should strive not to lean on the compiler to find silly errors: so build up working memory of syntax etc. and slow down to double check the code you wrote. But having that compilation and unit test loop - or even a repl - I could code as efficiently without it.
DrScump 6y ago
Plus, your organization had to pay for CPU cycles and I/Os. a la carte.
- jlokier 6y ago
  Which has come full circle, in the form of cloud per-minute, per-gigabyte rental.

afarrell 6y ago

It seems like that to me as well, but what is an engineer to do if he feels confused about what he's building on top of and he gets told that he's just feeling impostor syndrome or needs to stop "trying to understand the universe" or that his boss trusts that he does actually understand things? It has been true my whole career and I can't wait for the moment when I can retire and then have the time to learn enough to feel like I know what I'm doing.

Delk 6y ago
The "universe" is also, in some sense, much more complex than a couple of decades ago. In most cases we don't need to deal with as much low-level complexity or with cumbersome tooling, but "understanding the universe" would mean understanding pretty much all the levels of abstraction we have (and we have a lot), a diverse stack of technologies with rather different paradigms, a security landscape that has become much more complex with all the code we're running in a browser, and so on. I don't know what one is to do, but I somehow feel that "understanding the universe" is much more difficult nowadays than it perhaps was a few decades ago. There were probably fewer levels of abstraction, less diversity of technology, UIs might have been programmed in the same language as the rest of the program so there might not have been a need to juggle in a few languages at the same time, etc. Not to mention that since it's nowadays possible to program reasonably without having all that lower-level understanding, unlike perhaps a few decades ago, lots of people do that and (immediate) productivity expectations naturally grow to match. I feel the pain, though.
- alisonatwork 6y ago
  I agree that there were less levels of abstraction, but I think the domain in which each individual developer operates isn't that much different. In my first "serious" programming job, I was hired to be a C programmer. But just to get C building, you also need to understand Makefiles, and to understand those you need to understand shell script. We printed stuff to console in color, so we needed to understand ANSI escape sequences. We also printed to printers, so we needed to understand HP PCL. Some of our systems needed to talk to dedicated hardware attached on the serial port, so we had to learn proprietary protocols. Newer devices were attached via TCP, so we had to learn sockets and those protocols too. We had our own home-built database, but sometimes we needed SQL. We used Informix, later Postgres. Each had a different stored procedure language and C bindings. We deployed on UNIX, so we needed to understand files and pipes and processes and semaphores. We used a modem to dial our our customer sites, so needed to know about a tool called Kermit. There were many different flavors of UNIX back then, and they all had slightly different behaviors. We had to learn all the quirks that applied to the APIs we were using. Later on we started to build front ends in Java AWT and Python GTK, and that brought in a slew of new tech. Much of this tech was still pre-2000. In my current job, I am mostly working in Java and Python, but now they are backend languages. The services are built using Gradle, which needs knowledge of Groovy, or setuptools which is just more Python. The packages are bundled in containers and pushed to the cloud. There is a lot of engineering that enables this stack, but the only part we need to know about is the format of a Dockerfile (so, basic shell scripting) and various YAMLs for CloudFormation and the CI/CD system. We talk to lots of other services, but we don't need to know anything about TCP or sockets, we just need to use a library to manage the HTTP connections and another library to convert the wire format to objects. We don't care about presentation logic or reporting because someone elsewhere in the stack is going to do that. We define our contracts in Protobuf, JSON or XML and move on. Some services need to keep state, and for almost all use cases the file system is sufficient - in AWS that means S3. Sometimes we send or receive messages - in AWS that means SNS/SQS. We also push metrics and logging and distributed trace context, but that's just more HTTP abstracted away into a library. We build dashboards in Grafana using OpenTSDB query language. We check our logs in Kibana using Lucene query language. There is a lot to learn. I could spend even more time learning about Kubernetes or Kafka or Go or Redis or a hundred other components required by software outside my current domain, but none of that is necessary in my current role. Back in the day I could have spent time learning about OS/390 or Syslog or Perl or Berkeley DB too. So, although I agree that "the universe" in general is much more complex, I haven't found that to be true about the part individual developers are expected to understand.
alisonatwork 6y ago
At a certain point you need to trust that the library you are calling or the framework you are developing inside is doing what it is supposed to do. Going deeper than that can be interesting as a learning experience, and occasionally it's required to figure out really difficult bugs or performance issues, but it's not a prerequisite for being able to understand your own work. Identify the edges of your domain, accept them and treat them as the rules of the game. That said, by the sounds of it you have worked with some colleagues who have been unhelpful in teaching you the rules. This might be because you are building on a poorly-architected thing that cannot easily be explained, or in a system that does not have detailed documentation. Both of those situations are solvable problems, and neither of them is an excuse for the people who do understand to gatekeep their knowledge or discourage curiosity.

gregjor 6y ago

This was called “desk checking.” You had to learn to read code and actually understand what it was doing. You may only get a few chances a day to run your code. I remember the feeling submitting my job (a deck of punch cards) and then getting my cards and a small printout back in a few minutes because of an ABEND (abnormal end, or crash). I still do that, along with targeted experimenting on small bits of code I need to understand better. The skill of reading code and “running” it in the head seems forgotten now, though I think talented programmers usually get there.

alisonatwork 6y ago
One of my biggest disappointments in pair programming over the past few years is realizing how many developers are unable to explain what the code they just wrote actually does. Like, they can't do a verbal debug, stepping through each line and describing the expected state. Everyone copies from the best, but it seems a lot of developers just copy, paste and then let the IDE autofix any warnings. The code block becomes a magical incantation rather than a sequence of operations. It never occurred to me that the reason I look at code differently is simply because I am from am older generation. I only started coding professionally in the 90s, but even then compiling wasn't especially fast so it was expected to review code before kicking off a build. And as a kid in the 80s I spent more time reading code in books and magazines than actually at a computer typing it in, so I had to think about and imagine what it would do. There is definitely a lot to be said for the modern world of linters, unit tests etc running continuously - I wouldn't want to give up any of that up! But I do wonder if the developers who aren't able to explain what their code actually does are missing some fundamental skill. Clearly it doesn't matter to the customer because these developers still produce working products, but it feels to me like some level of quality has been lost.
- gregjor 6y ago
  My experience as well. Combined with the irresistible attraction to new and increasingly complex frameworks, tools, and infrastructure, this leads to a lot of failed projects. As an aging “not a good culture fit” programmer I found a niche taking on, finishing, and supporting the code left behind by developers who got in over their head and didn’t really understand what their code does.

quickthrower2 6y ago

I’d find trying to create a perfect program pre-compile and pre-unit test horribly inefficient. OTOH I think programmers should strive not to lean on the compiler to find silly errors: so build up working memory of syntax etc. and slow down to double check the code you wrote. But having that compilation and unit test loop - or even a repl - I could code as efficiently without it.

DrScump 6y ago

Plus, your organization had to pay for CPU cycles and I/Os. a la carte.

jlokier 6y ago
Which has come full circle, in the form of cloud per-minute, per-gigabyte rental.