5 ms·

>Theres no rhyme or reason to it at the API level. You have to look under the hood... Every abstraction is leaky. I have found that the best engineers are thos

>Theres no rhyme or reason to it at the API level. You have to look under the hood... Every abstraction is leaky. I have found that the best engineers are those who understand how things work under the hood. They can solve hard problems when they come up. People who rely upon a mountain of libraries are stumped and begin trying random things until something works. Honestly, what is the problem with understanding how your tools work? I don't know of any other engineering profession that thinks this way.

crimsonalucard 7y ago
>Honestly, what is the problem with understanding how your tools work? I don't know of any other engineering profession that thinks this way. There is actually a huge problem with this. Human intelligence is limited and therefore much of the art and science of computer programming is targeted towards managing complexity. The fundamental concept that humans use to reduce complexity in everything from math, physics, language, chemistry and everything out there is abstraction. When look at a wall you are looking at an abstraction. To truly understand the nature of the wall you must understand that it is composed of plant cells, (wood) which is composed of carbon based molecules which at is center is all serialized as a single hereditary molecule called DNA which is itself composed of atoms that are governed by complex differential equations and on and on and on... You don't need to know any of this to understand the wall because of abstraction. You encapsulate all that complexity into a concept called a wall. Programming is largely the same thing, with abstraction made explicit. To move forward, to produce things of greater complexity we have to move in the direction of good, non-leaky abstractions. Limitations in our intelligence will inevitably force us to become ignorant of the underlying implementations of many technologies so the abstractions around these things must be excellent. It is the only way forward and it is also arguably the reason why humans are different from other animals. We have the ability to abstract. This concept is everywhere in computer science. The web programmer doesn't need to understand assembly language even though it's the underlying implementation of all his tools. But you are right in the sense that it doesn't doesn't mean a programmer can't learn assembly. But this still illustrates my point. HUmans have a limited ability in handling complexity and therefore we must use abstraction to handle it. In fact, there is an area of computer science where this has become startlingly obvious. It involves machine learning. Take a visualization algorithm that can recognize objects. Currently the de facto way of doing this training a black box of neural networks. The complexity is so high that we have no idea how to program such algorithms by hand and the only way forward is to rely on these black box abstractions where we "train" the system. Even if the programmer pries open the neural network and tries to understand what each neuron is doing in the context of other neurons is impossible. In short. To build systems of greater complexity developers must use tools to abstract complexity. SQL has a different if not tangential problem. It's not leaking enough complexity. It's hiding too much, and producing an interface that is expensive in terms of performance. I'm proposing an abstraction that is one to one. Where if I learn the language and I understand how to optimize the language then a bijection allows me to optimize the underlying implementation as well without understanding it. To borrow the term from Rust and C++: Zero Cost abstractions.
- EpicEng 7y ago
  A database is not on the same level as e.g. the CPU. Almost none of us could write assembler for a modern chip without a lot initial of research. That's ok because we have compilers, the OS, and other layers to handle that for us. However, even for your bog standard web dev, a database is up at the top of the list of systems you need to understand in order to do your job. If you have no clue how to scale it out, how to optimize queries, hell, even how to write non-trivial queries (!), you're not doing your job. You sit there and write an N+1 on every page via your favorite ORM and no one knows why everything is so slow. A carpenter doesn't have to be a blacksmith, but they do need to know how a saw works. Not knowing the basics of how a DB works makes you disposable, just another JS/HTML/CSS monkey. Can't solve the hard problems, but has no problem bloating every page they create beyond reason.
  - crimsonalucard 7y ago
    >That's ok because we have compilers, the OS, and other layers to handle that for us. This is the essence of my argument. Why can't the database API occupy a layer similar to the OS or compilers? If I don't understand the underlying implementation of compilers and OSs' why do I need to understand the underlying implementation of my database? Why is the OS abstraction different from the DB sql abstraction? Can't we write abstractions that are zero cost? The answer is we can, however we haven't and therefore the only solutions available to us is to use SQL and understand the lower level details of the database. >However, even for your bog typical web dev, a database is right up there in the list of systems you need to understand in order to do your job. I'm not talking about your responsibilities on the job. I'm questioning the current state of tech. A web developer uses an API exposed by an OS and doesn't have to understand the OS. A web developer uses an API exposed by the compiler and doesn't have to understand Assembly. A web developer uses an API exposed by a database but he MUST understand the underlying features of a database. See the difference? I'm pointing out an issue that could be improved not what you need to know for your job. >A carpenter doesn't have to be a blacksmith, but they do need to know how a saw works. SQL is a saw that breaks in half on every use. In order to use such a saw a carpenter must have the knowledge of a blacksmith to reforge the metal and combine the two halves of the saw into a single metal blade. A leaky abstraction. That is a proper analogy for SQL.

crimsonalucard 7y ago

>Honestly, what is the problem with understanding how your tools work? I don't know of any other engineering profession that thinks this way. There is actually a huge problem with this. Human intelligence is limited and therefore much of the art and science of computer programming is targeted towards managing complexity. The fundamental concept that humans use to reduce complexity in everything from math, physics, language, chemistry and everything out there is abstraction. When look at a wall you are looking at an abstraction. To truly understand the nature of the wall you must understand that it is composed of plant cells, (wood) which is composed of carbon based molecules which at is center is all serialized as a single hereditary molecule called DNA which is itself composed of atoms that are governed by complex differential equations and on and on and on... You don't need to know any of this to understand the wall because of abstraction. You encapsulate all that complexity into a concept called a wall. Programming is largely the same thing, with abstraction made explicit. To move forward, to produce things of greater complexity we have to move in the direction of good, non-leaky abstractions. Limitations in our intelligence will inevitably force us to become ignorant of the underlying implementations of many technologies so the abstractions around these things must be excellent. It is the only way forward and it is also arguably the reason why humans are different from other animals. We have the ability to abstract. This concept is everywhere in computer science. The web programmer doesn't need to understand assembly language even though it's the underlying implementation of all his tools. But you are right in the sense that it doesn't doesn't mean a programmer can't learn assembly. But this still illustrates my point. HUmans have a limited ability in handling complexity and therefore we must use abstraction to handle it. In fact, there is an area of computer science where this has become startlingly obvious. It involves machine learning. Take a visualization algorithm that can recognize objects. Currently the de facto way of doing this training a black box of neural networks. The complexity is so high that we have no idea how to program such algorithms by hand and the only way forward is to rely on these black box abstractions where we "train" the system. Even if the programmer pries open the neural network and tries to understand what each neuron is doing in the context of other neurons is impossible. In short. To build systems of greater complexity developers must use tools to abstract complexity. SQL has a different if not tangential problem. It's not leaking enough complexity. It's hiding too much, and producing an interface that is expensive in terms of performance. I'm proposing an abstraction that is one to one. Where if I learn the language and I understand how to optimize the language then a bijection allows me to optimize the underlying implementation as well without understanding it. To borrow the term from Rust and C++: Zero Cost abstractions.

EpicEng 7y ago
A database is not on the same level as e.g. the CPU. Almost none of us could write assembler for a modern chip without a lot initial of research. That's ok because we have compilers, the OS, and other layers to handle that for us. However, even for your bog standard web dev, a database is up at the top of the list of systems you need to understand in order to do your job. If you have no clue how to scale it out, how to optimize queries, hell, even how to write non-trivial queries (!), you're not doing your job. You sit there and write an N+1 on every page via your favorite ORM and no one knows why everything is so slow. A carpenter doesn't have to be a blacksmith, but they do need to know how a saw works. Not knowing the basics of how a DB works makes you disposable, just another JS/HTML/CSS monkey. Can't solve the hard problems, but has no problem bloating every page they create beyond reason.
- crimsonalucard 7y ago
  >That's ok because we have compilers, the OS, and other layers to handle that for us. This is the essence of my argument. Why can't the database API occupy a layer similar to the OS or compilers? If I don't understand the underlying implementation of compilers and OSs' why do I need to understand the underlying implementation of my database? Why is the OS abstraction different from the DB sql abstraction? Can't we write abstractions that are zero cost? The answer is we can, however we haven't and therefore the only solutions available to us is to use SQL and understand the lower level details of the database. >However, even for your bog typical web dev, a database is right up there in the list of systems you need to understand in order to do your job. I'm not talking about your responsibilities on the job. I'm questioning the current state of tech. A web developer uses an API exposed by an OS and doesn't have to understand the OS. A web developer uses an API exposed by the compiler and doesn't have to understand Assembly. A web developer uses an API exposed by a database but he MUST understand the underlying features of a database. See the difference? I'm pointing out an issue that could be improved not what you need to know for your job. >A carpenter doesn't have to be a blacksmith, but they do need to know how a saw works. SQL is a saw that breaks in half on every use. In order to use such a saw a carpenter must have the knowledge of a blacksmith to reforge the metal and combine the two halves of the saw into a single metal blade. A leaky abstraction. That is a proper analogy for SQL.