5 ms·

Anyone have examples of 'good' abstraction vs. 'bad' abstraction? Edit: Missed ntoshev's comment.

andreyf 17y ago
I had examples by cut them in fear of starting tribal conflicts, and because brand-name examples detract from the logical point. The ones I cut are: - JVM is a "bad" one, as most Java programmers aren't intended to ever think about JVM internals. Languages developed on top of Parrot and LLVM are "good" ones, as they spend considerable effort making their internals accessible. - ORMs on top of *SQL are "bad" if the user doesn't understand their implementation. Flat files and MongoDB are "good", assuming the user does (as I imagine most do). - TCP/IP used as a byte stream is "bad", with an understanding of packet ordering/repetition/correctness is "good". Rails' templates are transparent abstractions to HTML compared to some others, but a lot of the other "magic" is an example of some pretty opaque stuff - the ORM being an example, and also abstraction from the network layer.
- ntoshev 17y ago
  "Transparent" and "opaque" seem better terms than "good" and "bad". ORMs usually let you fall back to SQL if you need to (ActiveRecord does), so I would classify them as transparent. Relational databases themselves are opaque though, they basically present you with a subset of relational algebra and how its operations will be carried out depends on a myriad of parameters in a non-trivial way. And you cannot possibly push some little modification in the inner loop. Another good example is Larry Page asking how to set a custom user-agent for what became the googlebot in the then-current-JDK (and having no way to do it): http://groups.google.com/group/comp.lang.java/browse_thread/thread/6923c024ed392c85/88fa10845061c8ba http://groups.google.com/group/comp.lang.java/browse_thread/...
  - andreyf 17y ago
    Re: ORM/SQL, compare Django's ORM/MySQL to how PG implemented storage in news.YC - I've devoted several orders of magnitude more time working with the former, but I wouldn't be able to describe what happens in-hardware nearly as well as I could on the latter, which I read once (in a language I never seriously used) nearly a year ago.
    - ntoshev 17y ago
      I don't have experience with your specific example, but it seems you are complaining against the opacity of relational databases, not the ORM itself. I agree, relational databases are very opaque.
      - andreyf 17y ago
        If an ORM user can be fairly confident fairly quickly about the SQL generated from each of his ORM calls, it's a good abstraction. In my experience with Django's ORM, that hasn't been the case. Fault may be mine :)
      - nwatson 17y ago
        RDBMSs aren't that opaque ... A relational database exists to store rows of data in tables and to relate them, making CRUD operations possible in this context. At that level it's very abstract. But it also starts to expose some implementation details as appropriate, and that's why some DBAs make a very good living (I'm not a DBA myself). Transactions are required once there can be more than one DB client (well, even before that when considering atomicity/durability). And most DBs expose different transaction isolation levels because the various options have different speed vs semantic reliability implications (e.g. dirty reads). These make sense only in light of lower-level concepts that leak through from the notion that data is stored on disk, buffered in memory, handled by threads, protected by locks, etc. And then there are details of indexing, clustering, distributing table data among available disks, where to store the transaction log, how to secure/encrypt data on disk, disaster recovery, support for high availability, etc. These all are exposed leaks that relate to what the little heads on the disk and CPUs and network all do in light of your desired performance and reliability. Some DBs also abstract out the back end, where you can plug in different low-level storage engines or even expose your own data structures as virtual tables (e.g. flat files or network data streams). You're able to write your own back end if you want. And I've heard of at least one effort to rewrite RDBMSs to run well on and take advantage of the nature of SSDs vs. hard drives -- I'll be curious as to how that turns out. I also believe the nature of such a system is fundamentally different enough to make adapting existing RDBMSs to SSDs very difficult. I'd say then that many RDBMSs have done a very good job exposing the low detail levels to the level that's appropriate.
- jedbrown 17y ago
  Making internals transparent is a very bad thing when it encourages users to write code that relies on implementation details. On the other hand, a certain amount of transparency is critical so that users can easily develop a mental performance model, and so that it is clear how they can extend the abstraction (e.g. by writing plugins) when necessary. One of the most difficult parts of library design is to have enough of the good transparency without ending up with a brittle system or a system that users have to wrap in order to break awkward dependencies.
DannoHung 17y ago
A good abstraction is one that does exactly what it says it will, no more and no less. I believe that this is an exceedingly difficult goal to meet.
dpatru 17y ago
A good abstraction is one that makes it easy to express the solution in a way that the customer wants. A bad abstraction makes it hard. For example, if the customer in an end-user and wants a table-like presentation, using an HTML table is a good abstraction. If your customer is a css designer who just wants to see divs, an HTML table may not be a good abstraction.

I had examples by cut them in fear of starting tribal conflicts, and because brand-name examples detract from the logical point. The ones I cut are: - JVM is a "bad" one, as most Java programmers aren't intended to ever think about JVM internals. Languages developed on top of Parrot and LLVM are "good" ones, as they spend considerable effort making their internals accessible. - ORMs on top of *SQL are "bad" if the user doesn't understand their implementation. Flat files and MongoDB are "good", assuming the user does (as I imagine most do). - TCP/IP used as a byte stream is "bad", with an understanding of packet ordering/repetition/correctness is "good". Rails' templates are transparent abstractions to HTML compared to some others, but a lot of the other "magic" is an example of some pretty opaque stuff - the ORM being an example, and also abstraction from the network layer.

ntoshev 17y ago
"Transparent" and "opaque" seem better terms than "good" and "bad". ORMs usually let you fall back to SQL if you need to (ActiveRecord does), so I would classify them as transparent. Relational databases themselves are opaque though, they basically present you with a subset of relational algebra and how its operations will be carried out depends on a myriad of parameters in a non-trivial way. And you cannot possibly push some little modification in the inner loop. Another good example is Larry Page asking how to set a custom user-agent for what became the googlebot in the then-current-JDK (and having no way to do it): http://groups.google.com/group/comp.lang.java/browse_thread/thread/6923c024ed392c85/88fa10845061c8ba http://groups.google.com/group/comp.lang.java/browse_thread/...
- andreyf 17y ago
  Re: ORM/SQL, compare Django's ORM/MySQL to how PG implemented storage in news.YC - I've devoted several orders of magnitude more time working with the former, but I wouldn't be able to describe what happens in-hardware nearly as well as I could on the latter, which I read once (in a language I never seriously used) nearly a year ago.
  - ntoshev 17y ago
    I don't have experience with your specific example, but it seems you are complaining against the opacity of relational databases, not the ORM itself. I agree, relational databases are very opaque.
    - andreyf 17y ago
      If an ORM user can be fairly confident fairly quickly about the SQL generated from each of his ORM calls, it's a good abstraction. In my experience with Django's ORM, that hasn't been the case. Fault may be mine :)
    - nwatson 17y ago
      RDBMSs aren't that opaque ... A relational database exists to store rows of data in tables and to relate them, making CRUD operations possible in this context. At that level it's very abstract. But it also starts to expose some implementation details as appropriate, and that's why some DBAs make a very good living (I'm not a DBA myself). Transactions are required once there can be more than one DB client (well, even before that when considering atomicity/durability). And most DBs expose different transaction isolation levels because the various options have different speed vs semantic reliability implications (e.g. dirty reads). These make sense only in light of lower-level concepts that leak through from the notion that data is stored on disk, buffered in memory, handled by threads, protected by locks, etc. And then there are details of indexing, clustering, distributing table data among available disks, where to store the transaction log, how to secure/encrypt data on disk, disaster recovery, support for high availability, etc. These all are exposed leaks that relate to what the little heads on the disk and CPUs and network all do in light of your desired performance and reliability. Some DBs also abstract out the back end, where you can plug in different low-level storage engines or even expose your own data structures as virtual tables (e.g. flat files or network data streams). You're able to write your own back end if you want. And I've heard of at least one effort to rewrite RDBMSs to run well on and take advantage of the nature of SSDs vs. hard drives -- I'll be curious as to how that turns out. I also believe the nature of such a system is fundamentally different enough to make adapting existing RDBMSs to SSDs very difficult. I'd say then that many RDBMSs have done a very good job exposing the low detail levels to the level that's appropriate.
jedbrown 17y ago
Making internals transparent is a very bad thing when it encourages users to write code that relies on implementation details. On the other hand, a certain amount of transparency is critical so that users can easily develop a mental performance model, and so that it is clear how they can extend the abstraction (e.g. by writing plugins) when necessary. One of the most difficult parts of library design is to have enough of the good transparency without ending up with a brittle system or a system that users have to wrap in order to break awkward dependencies.

DannoHung 17y ago

A good abstraction is one that does exactly what it says it will, no more and no less. I believe that this is an exceedingly difficult goal to meet.

dpatru 17y ago

A good abstraction is one that makes it easy to express the solution in a way that the customer wants. A bad abstraction makes it hard. For example, if the customer in an end-user and wants a table-like presentation, using an HTML table is a good abstraction. If your customer is a css designer who just wants to see divs, an HTML table may not be a good abstraction.