5 ms·

I've got a phrase for when I see people trying to "abstract" over certain types of problems by writing absurdly complicated code that ultimately saves negative

by Verdex 5y ago

I've got a phrase for when I see people trying to "abstract" over certain types of problems by writing absurdly complicated code that ultimately saves negative time. "Data abstraction is impossible." You can abstract an algorithm. Like, how you sort. Merge sort, bubble sort, etc. Normally it doesn't matter to the rest of the code, you just want a sorted list. You can abstract a type or an object. Hey, given some type T and some function T -> int, you can get an int. You CAN'T abstract a chunk of data which has semantically distinct members inside of it. If someone sends you a structure that has a field in it called "name", then you have no idea what that actually means. You have to get them to tell you what they mean by "name". The hard part being that inevitably "name" means one thing unless the "aux" field is set to a number that evenly divides the "misc" field at which point "name" has to be filtered by some arcane rules. This is why we have lists like "falsehoods programmers believe about time|names|etc". Time means different things for different applications. Names mean different things to different contexts and cultures. You have to find someone to tell you what all of the individual pieces of the object's internals means to them. no-code will probably see some success in niche domains where things have been fixed for years. Ultimately, many applications will need hand written code to handle all of the meaningful sub-components of the data being interacted with.

closeparen 5y ago
On the other hand some of the most powerful and widely used programs are those that operate on byte arrays with schemas in-band (the web) or otherwise specified at runtime (SQL). Imagine if you needed to write C structs in Nginx or Postgres corresponding to all the business objects and data types they’re going to touch. Worse, imagine you need to thread them through all the implementation! Many business applications are written that way. I think there are opportunities on the table to stop doing it.
- Verdex 5y ago
  So. There's json parsers, serde, row polymorphism, anon structs, macros, monads, and dictionaries. They all do the same type of thing, but that isn't abstracting data. That's abstracting the population of data or the serializing of it. However none of that is ever going to know that the name field needs to strip out the first three characters, rot 13, and then parse as a guid. Someone still needs to know what the data you're getting means. And there's no way to shake that. Although people do try and then we get lists like: things programmers don't understand about names.
jrumbut 5y ago
> "Data abstraction is impossible." That's an amazing point. It explains where the object oriented proponents of decades past went wrong in thinking there would be vendors for "Car" or "Employee" objects.
- twic 5y ago
  No, because you can absolutely abstract over data if it is encapsulated, which is the whole trick of objects, You can use a Point from any vendor, without caring if it has x and y, or r and theta, or some other representation inside, as long as it correctly implements the specified interface.
  - jrumbut 5y ago
    But very few interesting real world objects can be consistently described across different pieces of software like a Point can, which is where it ties into the article.
    - Verdex 5y ago
      Yeah, very much this. Not to mention, if you want to serialize Point, then you have to know what its internals look like and what they mean. There is no data abstraction. [Okay, so when you serialize Point, you can also serialize all code that will ever do anything with point. However, we generally don't do this (partially because of security issues). But it doesn't really help that much because whatever the serialized code produces when run has to be some datatype where do understand the semantic meanings behind its internals.]
    - deleted 5y ago
      [deleted]
  - Verdex 5y ago
    This feels like what you are describing is algorithmic or type abstraction. For your example, it's something like: Given some type Point which (I guess) interacts with some type Graph you should be able to run the Plot method. Or something. And in this case, yeah you can abstract, but it's not what I'm calling "data abstraction." [Maybe my name is just not quite right.] Data abstraction would be getting some arbitrary Point object and then being able to look into the internals and do the right thing. Whether it's x and y or r and theta. Now you might say, "that's just bad programming if you do that," but the point is that often times we don't have a choice in the matter because either of the constraints of the project OR because the "do the right thing" part interacts with non-code. So for example, if you want to send your Point over the network, then you either 1) Need to know the internals, what they mean, and how they work or 2) have to send along code for the remote device to execute to handle your custom internals. Case 2 is generally not done (and presents some security issues regardless). But this isn't just a technical issue. Names are notoriously difficult because data abstraction doesn't exist. How many first, middle, and last names does someone have? Is there a canonical ordering for the names? Does someone always have a name? What do the prefixes mean? What do the postfixes mean? These questions and more all have to do with what you're trying to use the name for AND what culture the name comes from. There is nothing you can do to abstract away the problem of names. And if you try you'll just end up with a lot of messy, complex code, that breaks all the time.
ta988 5y ago
This is why we have ontologies. Still at a early stage of data abstraction but it is getting there.
fsckboy 5y ago
> "Data abstraction is impossible." Data abstraction is hard and many people are no good at it, but it's not impossible; it can't be, because it's required. don't think of data abstraction as one and done, think of it as a process. Maintaining (as in, maintenance-ing rather than obeying) your data abstractions as you make modifications to code is the key to clean code, it's where the real refactoring takes place. if your brain works like mine does, it is paralyzing to work on code without data abstractions, like always thinking of a staircase as all of its component parts rather than as a staircase
phaedrus 5y ago
...and the corollary to that is turning things into data is often a much simpler alternative to abstraction. For example suppose you have a program with configuration. You might have a "ConfigurationProvider" and abstract that with an "IConfigurationProvider" and mock implementations of that interface for unit tests. Or you could simply define a Plain Old Data structure with fields for the configuration data. Then you can construct an instance of that data structure in different ways as appropriate. The key idea there is: "data" is something that's already automatically decoupled from where the data came from. For code you have to do work to achieve that decoupling, and it will always leak.

by Verdex 5y ago

closeparen 5y ago

On the other hand some of the most powerful and widely used programs are those that operate on byte arrays with schemas in-band (the web) or otherwise specified at runtime (SQL). Imagine if you needed to write C structs in Nginx or Postgres corresponding to all the business objects and data types they’re going to touch. Worse, imagine you need to thread them through all the implementation! Many business applications are written that way. I think there are opportunities on the table to stop doing it.

Verdex 5y ago
So. There's json parsers, serde, row polymorphism, anon structs, macros, monads, and dictionaries. They all do the same type of thing, but that isn't abstracting data. That's abstracting the population of data or the serializing of it. However none of that is ever going to know that the name field needs to strip out the first three characters, rot 13, and then parse as a guid. Someone still needs to know what the data you're getting means. And there's no way to shake that. Although people do try and then we get lists like: things programmers don't understand about names.

jrumbut 5y ago

> "Data abstraction is impossible." That's an amazing point. It explains where the object oriented proponents of decades past went wrong in thinking there would be vendors for "Car" or "Employee" objects.

twic 5y ago
No, because you can absolutely abstract over data if it is encapsulated, which is the whole trick of objects, You can use a Point from any vendor, without caring if it has x and y, or r and theta, or some other representation inside, as long as it correctly implements the specified interface.
- jrumbut 5y ago
  But very few interesting real world objects can be consistently described across different pieces of software like a Point can, which is where it ties into the article.
  - Verdex 5y ago
    Yeah, very much this. Not to mention, if you want to serialize Point, then you have to know what its internals look like and what they mean. There is no data abstraction. [Okay, so when you serialize Point, you can also serialize all code that will ever do anything with point. However, we generally don't do this (partially because of security issues). But it doesn't really help that much because whatever the serialized code produces when run has to be some datatype where do understand the semantic meanings behind its internals.]
  - deleted 5y ago
    [deleted]
- Verdex 5y ago
  This feels like what you are describing is algorithmic or type abstraction. For your example, it's something like: Given some type Point which (I guess) interacts with some type Graph you should be able to run the Plot method. Or something. And in this case, yeah you can abstract, but it's not what I'm calling "data abstraction." [Maybe my name is just not quite right.] Data abstraction would be getting some arbitrary Point object and then being able to look into the internals and do the right thing. Whether it's x and y or r and theta. Now you might say, "that's just bad programming if you do that," but the point is that often times we don't have a choice in the matter because either of the constraints of the project OR because the "do the right thing" part interacts with non-code. So for example, if you want to send your Point over the network, then you either 1) Need to know the internals, what they mean, and how they work or 2) have to send along code for the remote device to execute to handle your custom internals. Case 2 is generally not done (and presents some security issues regardless). But this isn't just a technical issue. Names are notoriously difficult because data abstraction doesn't exist. How many first, middle, and last names does someone have? Is there a canonical ordering for the names? Does someone always have a name? What do the prefixes mean? What do the postfixes mean? These questions and more all have to do with what you're trying to use the name for AND what culture the name comes from. There is nothing you can do to abstract away the problem of names. And if you try you'll just end up with a lot of messy, complex code, that breaks all the time.

ta988 5y ago

This is why we have ontologies. Still at a early stage of data abstraction but it is getting there.

fsckboy 5y ago

> "Data abstraction is impossible." Data abstraction is hard and many people are no good at it, but it's not impossible; it can't be, because it's required. don't think of data abstraction as one and done, think of it as a process. Maintaining (as in, maintenance-ing rather than obeying) your data abstractions as you make modifications to code is the key to clean code, it's where the real refactoring takes place. if your brain works like mine does, it is paralyzing to work on code without data abstractions, like always thinking of a staircase as all of its component parts rather than as a staircase

phaedrus 5y ago

...and the corollary to that is turning things into data is often a much simpler alternative to abstraction. For example suppose you have a program with configuration. You might have a "ConfigurationProvider" and abstract that with an "IConfigurationProvider" and mock implementations of that interface for unit tests. Or you could simply define a Plain Old Data structure with fields for the configuration data. Then you can construct an instance of that data structure in different ways as appropriate. The key idea there is: "data" is something that's already automatically decoupled from where the data came from. For code you have to do work to achieve that decoupling, and it will always leak.