3 ms·

Your prof was onto something that seems to be very in the zeitgeist today. To "understood how to represent your data" you have to understand what it is you're t

by sedachv 14y ago

Your prof was onto something that seems to be very in the zeitgeist today. To "understood how to represent your data" you have to understand what it is you're trying to represent. Eric Evans popularized this notion with Domain-Driven Design. If you follow this line of thinking far enough, you realize that computer programming is just applied analytic philosophy. You have your metamodel (logic/programming language) and then you build your model (ontology/software). I really like your assembly line metaphor. Knowing that a "customer" has a name and email address is almost of no importance compared to understanding how the "customer" information arrives, the actions around the "customer," and the end result of the actions. That's the assembly line.

robomartin 14y ago
I think that, to some degree, becoming an effective APL programmer almost required becoming good at data representation. If you want to do things the "APL way" you have to think about structures that work well with array, matrix and tensor processing ideas. When you could represent your data in the most optimal way you could sometimes write a function with only a few operators that could do what needed to be done to that data. This means that, to some extent, data representation might have several local minima (in terms of being optimal) with each population of possible representations exists around the language or toolset you are going to use to process this data. What I mean is that the optimal data representation for an embedded system (as an example) written in assembler might be different than that of the same system written in C or Forth. I don't use APL very much any more. Back when I did I was also programming in Forth, C and Assembler. I know that the decisions I would make --in terms of optimal data representation-- varied from language to language. This is partly true because each language, in my case, was being used to deal with different problem domains. For example, APL could do high-level computations while Forth was great for low-level, real-time stuff.
- 0ack 14y ago
  Interesting comments. I've been working with an APL-derivative and this sounds similar to how I have been thinking about things. It's a matter of massaging the data to get in into a certain representation, e.g. a matrix, then from there it's very, very easy to work with using APL. As you say a few operators is all you need. The code is very terse. Very powerful. The real work seems to be shaping the data into the right representation first, a matrix. I use non-APL-descendent programs, the usual UNIX utilities, to do this for now.
  - robomartin 14y ago
    Right. I remember having fun with this doing an APL application to aid in DNA sequencing. There were a number of ways to represent the data provided by the sequencer. At one point it almost became a game to see how small of an expression one could create to process the data by changing the representation. With APL one has to be careful not to create monsters that cause geometric expansion in memory needs. If the data set is large and the expressions processing the data cause frequent expansion into matrices or tensors (n-dimensional data structures where n > 2) one could end-up with geometric or exponential memory requirements. This, again, is another case of having to understand and fit data representation to the programming language AND the approach one will use to work with the data. While languages like APL can be great, they can be disastrous in the hands of a programmer who does not understand what might be going on at a lower level. Sometimes there's nothing better than good-old low-level C.

by sedachv 14y ago

robomartin 14y ago

I think that, to some degree, becoming an effective APL programmer almost required becoming good at data representation. If you want to do things the "APL way" you have to think about structures that work well with array, matrix and tensor processing ideas. When you could represent your data in the most optimal way you could sometimes write a function with only a few operators that could do what needed to be done to that data. This means that, to some extent, data representation might have several local minima (in terms of being optimal) with each population of possible representations exists around the language or toolset you are going to use to process this data. What I mean is that the optimal data representation for an embedded system (as an example) written in assembler might be different than that of the same system written in C or Forth. I don't use APL very much any more. Back when I did I was also programming in Forth, C and Assembler. I know that the decisions I would make --in terms of optimal data representation-- varied from language to language. This is partly true because each language, in my case, was being used to deal with different problem domains. For example, APL could do high-level computations while Forth was great for low-level, real-time stuff.

0ack 14y ago
Interesting comments. I've been working with an APL-derivative and this sounds similar to how I have been thinking about things. It's a matter of massaging the data to get in into a certain representation, e.g. a matrix, then from there it's very, very easy to work with using APL. As you say a few operators is all you need. The code is very terse. Very powerful. The real work seems to be shaping the data into the right representation first, a matrix. I use non-APL-descendent programs, the usual UNIX utilities, to do this for now.
- robomartin 14y ago
  Right. I remember having fun with this doing an APL application to aid in DNA sequencing. There were a number of ways to represent the data provided by the sequencer. At one point it almost became a game to see how small of an expression one could create to process the data by changing the representation. With APL one has to be careful not to create monsters that cause geometric expansion in memory needs. If the data set is large and the expressions processing the data cause frequent expansion into matrices or tensors (n-dimensional data structures where n > 2) one could end-up with geometric or exponential memory requirements. This, again, is another case of having to understand and fit data representation to the programming language AND the approach one will use to work with the data. While languages like APL can be great, they can be disastrous in the hands of a programmer who does not understand what might be going on at a lower level. Sometimes there's nothing better than good-old low-level C.