5 ms·

Are Databases and Functional Programming at odds?

26 points by Cieplak 14y ago

wulczer 14y ago
What? Relational databases and functional programming are very much aligned, see http://thoughts.j-davis.com/2011/09/25/sql-the-successful-cousin-of-haskell/ http://thoughts.j-davis.com/2011/09/25/sql-the-successful-co... (that's the second time I link to that post [I'm not the author] on HN, because I think it's awesome).
gaius 14y ago
The only "issue" is that many functional languages don't have good database bindings, once you have the data it's very natural to work on a resultset as a list (of tuples/records/whatever you prefer to call them). Now might be a good time to plug the Oracle bindings for OCaml: http://gaiustech.github.com/ociml/ http://gaiustech.github.com/ociml/
- Roboprog 14y ago
  Works for me! I would often prefer to avoid the ORM make-work, and just take a Perl-ish approach (in Java), processing something like List <Map <String, Object> > (emulate an array/list of hash/tuple), rather than dealing with a bunch of boiler plate classes which contain no logic what-so-ever.
jerf 14y ago
Functional languages aren't stateless... at least not the way this person, and many others, imagine. Of course they have state, and manipulate state, and do anything else that any other language does. Therefore, they have no conceptual problem with talking to databases any more than they do talking to anything else along a socket (or byte stream). The real get-to-the-root-problem answer, which I don't see on StackOverflow right now, is to come to a better understanding of what functional programmers mean by statelessness. Broadly: A function ought to be a function. That is, when called with the same parameters, it should return the same result. A functional programmer says that in an imperative program, there is an implicit additional parameter passed to every function, call it World, which contains vast quantities of state, global variables, global references, the IO state of the world, the ability to open sockets, etc., and that every function implicitly returns a new World value with unbounded changes made to it. Functional programming removes this implicit passing of the World value, and replaces it with an explicit version. (Or rather, the strong, modern version of "Functional Programming", which is implicitly what was specified when talking about the importance of statelessness.) In Haskell, this is the IO datatype used as a monad. A stateless function is one that does not examine World and depends solely on its non-World parameters. You can write them in non-functional languages, they just don't particularly provide you any assistance. Functional languages are ones that do provide you assistance, such as the State monadic value, which contrary to popular belief is in fact fully stateless when the arguments to all the relevant functions are de-sugared and examined. (It has a state-like effect as the programmer thinks about it, but is actually just syntax sugar for threading a value through a series of functions, as older-school functional languages like Erlang do.) Functional languages conceptually allow functions to apply to the real world by threading a World value through some functions, and ensuring that World values can't be reused. (How they literally function may differ in ways you don't care about for now.) This is what the IO monad is doing in Haskell, ensuring there's no way to reuse a given World psuedo-value. So, no, functional languages have no problems talking to the database when passed the relevant World value. (Hypothetically they could be passed a restricted value that would let them only talk to the database, rather than have full access to all IO.) You just won't be able to crack open a connection to the database and start talking to it unless you're in a context where that is permitted. You won't be allowed to do it without realizing it, declaring it, and explicitly controlling it. (Also, as is the way with this sort of post, there's some ways to quibble with it. I was trying to keep it not Haskell specific, but that sort of makes it hard to maintain consistent terminology, especially when speaking to novices. Perhaps I should just have gone for Haskell only.)
- Roboprog 14y ago
  I think another name for "World" is "DATA DIVISION", at least in some circles. Or at least it would be, if more Java programmers actually recognized the language idiom that they are really programming. (says the tired Java programmer)
- Roboprog 14y ago
  The entire "monad" discussion is pretty hard to get your head around. Perhaps part of the problem is that it represents not just side effects, but also forced ordering of operations (albeit, largely to allow for those effect-full interactions with the outside world). I think that maybe for "industrial" programmers, we need a language, perhaps something like Scala, where the default is pure functions and immutable data, but you can tag things (functions & data) as "mutants", er, "mutable"/"mutates", and those tags "taint" what those elements can be combined with. Call something with side effects from something that should be pure, and get an error, that sort of thing. Then, teach people to use mutable data as sparingly as possible. Maybe wrap database updates in some kind of (small!) message handling actor?
  - jerf 14y ago
    "Call something with side effects from something that should be pure, and get an error, that sort of thing." If you enforce that, you get Haskell, where you have no choice but to wrap your head around isolating your IO layer from everything else. If you don't enforce that, you don't get very much material difference in practice. I'm concerned that in practice, 99% purity isn't very useful. We shall see. (You get advantage from 0 - 50%, then I think a long plateau, then a sharp spike right at 100%. It can be hard to make the leap, since there's no way to ease into it.) "The entire "monad" discussion is pretty hard to get your head around. Perhaps part of the problem is that it represents not just side effects, but also forced ordering of operations (albeit, largely to allow for those effect-full interactions with the outside world)." Well, part of the problem is that "monad" doesn't mean any of those things. It's an interface (very much like the Java sense of the term) which has some implementations that enforce those properties, and other implementations that do not. Which is to say that yes, it is hard to get your mind wrapped around, but this is largely because of the spray of terrible, terrible explanations of the monad interface sprayed about by people who barely understand it themselves. The best way to actually learn them is Learn You a Haskell; if you've already learned basic Haskell syntax you can leap straight to chapter 11[1]. (Do not jump straight to the Monad chapter 12. Start at 11. Start earlier if you don't know the syntax.) "Monad" is an adjective, not a noun. [1]: http://learnyouahaskell.com/functors-applicative-functors-and-monoids http://learnyouahaskell.com/functors-applicative-functors-an...
WildUtah 14y ago
See "Out of the Tar Pit" by Moseley and Marks http://web.mac.com/ben_moseley/frp/paper-v1_01.pdf http://web.mac.com/ben_moseley/frp/paper-v1_01.pdf It's a great idea for a new paradigm of functional relational programming.
- bni 14y ago
  Thanks I have been looking for this text for a while, read it once and then lost it. I think it has some excellent insights, all developers should read it.
RyanMcGreal 14y ago
Don't feel bad: databases and object-oriented programming are also at odds. It's no accident that object-relational mappers are so leaky.
- bni 14y ago
  I agree. Especially if you are trying to have a 1:1 mapping between tables and classes. The convenience you gain in the short term will be paid by complexity, performance, and code base lock in 100 times in the middle and far term. The relational database and your imperative program are two different things, trying to "unify" them will not work.
- FuzzyDunlop 14y ago
  I'm of the opinion that the obsession with these ORMs is, in some small part, the result of an abject fear of using SQL directly. There's a certain incompatibility that prevents them being truly powerful without bending your schema to the will of the ORM, or without sacrificing a lot of the potential functionality SQL has to offer (and thus the readability of your code as you try to shoehorn these things in). I suppose NoSQL solutions help fill that niche, though.
kristianp 14y ago
Isn't SQL a functional language, of sorts? http://stackoverflow.com/questions/1153397/besides-a-declarative-language-is-sql-a-functional-language http://stackoverflow.com/questions/1153397/besides-a-declara...
- sp332 14y ago
  The answers on that page say "no".
- Dn_Ab 14y ago
  It is a declarative language. But the underlying math of SQL is that of relations. It's closer to prolog than to Haskell. I don't know much about the theory of SQL but at least I can draw on the analogy of deriving algorithms in a Relational setting vs a functional to emphasize that the essence of functions is not the same as the essence of relations: Unlike functions, relations can be nondeterministic and each relation has a converse.
- gaius 14y ago
  In PL/SQL a "function" in the FP sense would be called "deterministic".
michaelochurch 14y ago
Functional programming has to contend with "the real world", which is stateful. FP isn't about eliminating computational effects, but managing them. The virtue of functional programming is that most of what happens is stateless, so that when stateful effects are needed, they're simple, uncoupled, and do exactly what one expects them to do. One principle for these sorts of applications is the "command/query separation". In essence, a procedure exposed at an interface level should either (a) do something, and do exactly what the user expects, or (b) answer a question about the system without computational effects. The case of (a) might be called "side effects", but that's a misnomer in this case because the requested state changes are the main effects. Functional programming is really about having the most basic "building block" of a program be a referentially-transparent function, and to handle stateful effects either using an abstraction such as an Action DSL (which represents a collection of stateful behaviors, which are computed statelessly and then executed or "committed" in a stateful environment, relegating the statefulness to one "commit" function) or a monadic representation. In contrast, imperative programming uses computational actions as the atomic "building block". For small programs, either imperative or functional programming will work. Imperative programming isn't evil; sometimes (but not often) it's the right approach. The problem emerges as programs get larger and more complicated, and more hands pass over the code. If a function is referentially-transparent, its semantics are usually tractable. If proper tests exist for it, optimizing the function (e.g. for performance) shouldn't affect the rest of the system. The constraint to which the function's maintainer is held is that the semantics shouldn't change, and any alteration that does change its semantics is an objective bug. The function might be memoized (that's a private "side effect" that doesn't affect referential transparency) but it still has to behave the same way. That's a really good property. In an imperative language where functions are replaced by computational actions, these actions are frequently made more complex and featureful as time goes on, and unexpected interactions emerge. Semantics can be defined for imperative programs, but doing so is more complicated and it's rarely done in real-world professional programming. Most clusterfuck awful systems in software, for the record, are complex systems with imprecise semantics. I would say that 90% of software failures I've seen can be characterized in this way. This sort of thing can happen in any language, but it seems to be a lot more common in imperative programming, especially in verbose languages like Java or C++. It's just much harder to enforce precise semantics over an anything-goes imperative style of programming than a functional one. As for databases... well, reality is that purely functional programming isn't always possible. We're not just computing mathematical functions; we're trying to do things in the physical world. We need persistence, and that means telling a robotic arm to write bytes on magnetic media (stateful effect). There are several approaches that one can use. One that I like (and it has a lot of desirable properties in distributed systems) is the append-only approach: update and delete operations simply aren't allowed, although new records can shadow old ones in the context of time-sensitive queries. For example, the temperature in New York is 56 degrees now (8:30 am, May 7). In six hours, it might be 70. That doesn't mean we throw the old record away. It's still valid: the temperature was 56 at 8:30. The answer to the query, "What was the temperature at 8:30 in New York?" should never change. On the other hand, the answer to the query, "What is the most recent temperature in New York?" will change all the time.

26 points by Cieplak 14y ago

What? Relational databases and functional programming are very much aligned, see http://thoughts.j-davis.com/2011/09/25/sql-the-successful-cousin-of-haskell/ http://thoughts.j-davis.com/2011/09/25/sql-the-successful-co... (that's the second time I link to that post [I'm not the author] on HN, because I think it's awesome).

gaius 14y ago

The only "issue" is that many functional languages don't have good database bindings, once you have the data it's very natural to work on a resultset as a list (of tuples/records/whatever you prefer to call them). Now might be a good time to plug the Oracle bindings for OCaml: http://gaiustech.github.com/ociml/ http://gaiustech.github.com/ociml/

Roboprog 14y ago
Works for me! I would often prefer to avoid the ORM make-work, and just take a Perl-ish approach (in Java), processing something like List <Map <String, Object> > (emulate an array/list of hash/tuple), rather than dealing with a bunch of boiler plate classes which contain no logic what-so-ever.

jerf 14y ago

Functional languages aren't stateless... at least not the way this person, and many others, imagine. Of course they have state, and manipulate state, and do anything else that any other language does. Therefore, they have no conceptual problem with talking to databases any more than they do talking to anything else along a socket (or byte stream). The real get-to-the-root-problem answer, which I don't see on StackOverflow right now, is to come to a better understanding of what functional programmers mean by statelessness. Broadly: A function ought to be a function. That is, when called with the same parameters, it should return the same result. A functional programmer says that in an imperative program, there is an implicit additional parameter passed to every function, call it World, which contains vast quantities of state, global variables, global references, the IO state of the world, the ability to open sockets, etc., and that every function implicitly returns a new World value with unbounded changes made to it. Functional programming removes this implicit passing of the World value, and replaces it with an explicit version. (Or rather, the strong, modern version of "Functional Programming", which is implicitly what was specified when talking about the importance of statelessness.) In Haskell, this is the IO datatype used as a monad. A stateless function is one that does not examine World and depends solely on its non-World parameters. You can write them in non-functional languages, they just don't particularly provide you any assistance. Functional languages are ones that do provide you assistance, such as the State monadic value, which contrary to popular belief is in fact fully stateless when the arguments to all the relevant functions are de-sugared and examined. (It has a state-like effect as the programmer thinks about it, but is actually just syntax sugar for threading a value through a series of functions, as older-school functional languages like Erlang do.) Functional languages conceptually allow functions to apply to the real world by threading a World value through some functions, and ensuring that World values can't be reused. (How they literally function may differ in ways you don't care about for now.) This is what the IO monad is doing in Haskell, ensuring there's no way to reuse a given World psuedo-value. So, no, functional languages have no problems talking to the database when passed the relevant World value. (Hypothetically they could be passed a restricted value that would let them only talk to the database, rather than have full access to all IO.) You just won't be able to crack open a connection to the database and start talking to it unless you're in a context where that is permitted. You won't be allowed to do it without realizing it, declaring it, and explicitly controlling it. (Also, as is the way with this sort of post, there's some ways to quibble with it. I was trying to keep it not Haskell specific, but that sort of makes it hard to maintain consistent terminology, especially when speaking to novices. Perhaps I should just have gone for Haskell only.)

Roboprog 14y ago
I think another name for "World" is "DATA DIVISION", at least in some circles. Or at least it would be, if more Java programmers actually recognized the language idiom that they are really programming. (says the tired Java programmer)
Roboprog 14y ago
The entire "monad" discussion is pretty hard to get your head around. Perhaps part of the problem is that it represents not just side effects, but also forced ordering of operations (albeit, largely to allow for those effect-full interactions with the outside world). I think that maybe for "industrial" programmers, we need a language, perhaps something like Scala, where the default is pure functions and immutable data, but you can tag things (functions & data) as "mutants", er, "mutable"/"mutates", and those tags "taint" what those elements can be combined with. Call something with side effects from something that should be pure, and get an error, that sort of thing. Then, teach people to use mutable data as sparingly as possible. Maybe wrap database updates in some kind of (small!) message handling actor?
- jerf 14y ago
  "Call something with side effects from something that should be pure, and get an error, that sort of thing." If you enforce that, you get Haskell, where you have no choice but to wrap your head around isolating your IO layer from everything else. If you don't enforce that, you don't get very much material difference in practice. I'm concerned that in practice, 99% purity isn't very useful. We shall see. (You get advantage from 0 - 50%, then I think a long plateau, then a sharp spike right at 100%. It can be hard to make the leap, since there's no way to ease into it.) "The entire "monad" discussion is pretty hard to get your head around. Perhaps part of the problem is that it represents not just side effects, but also forced ordering of operations (albeit, largely to allow for those effect-full interactions with the outside world)." Well, part of the problem is that "monad" doesn't mean any of those things. It's an interface (very much like the Java sense of the term) which has some implementations that enforce those properties, and other implementations that do not. Which is to say that yes, it is hard to get your mind wrapped around, but this is largely because of the spray of terrible, terrible explanations of the monad interface sprayed about by people who barely understand it themselves. The best way to actually learn them is Learn You a Haskell; if you've already learned basic Haskell syntax you can leap straight to chapter 11[1]. (Do not jump straight to the Monad chapter 12. Start at 11. Start earlier if you don't know the syntax.) "Monad" is an adjective, not a noun. [1]: http://learnyouahaskell.com/functors-applicative-functors-and-monoids http://learnyouahaskell.com/functors-applicative-functors-an...

WildUtah 14y ago

See "Out of the Tar Pit" by Moseley and Marks http://web.mac.com/ben_moseley/frp/paper-v1_01.pdf http://web.mac.com/ben_moseley/frp/paper-v1_01.pdf It's a great idea for a new paradigm of functional relational programming.

bni 14y ago
Thanks I have been looking for this text for a while, read it once and then lost it. I think it has some excellent insights, all developers should read it.

RyanMcGreal 14y ago

Don't feel bad: databases and object-oriented programming are also at odds. It's no accident that object-relational mappers are so leaky.

bni 14y ago
I agree. Especially if you are trying to have a 1:1 mapping between tables and classes. The convenience you gain in the short term will be paid by complexity, performance, and code base lock in 100 times in the middle and far term. The relational database and your imperative program are two different things, trying to "unify" them will not work.
FuzzyDunlop 14y ago
I'm of the opinion that the obsession with these ORMs is, in some small part, the result of an abject fear of using SQL directly. There's a certain incompatibility that prevents them being truly powerful without bending your schema to the will of the ORM, or without sacrificing a lot of the potential functionality SQL has to offer (and thus the readability of your code as you try to shoehorn these things in). I suppose NoSQL solutions help fill that niche, though.

kristianp 14y ago

Isn't SQL a functional language, of sorts? http://stackoverflow.com/questions/1153397/besides-a-declarative-language-is-sql-a-functional-language http://stackoverflow.com/questions/1153397/besides-a-declara...

sp332 14y ago
The answers on that page say "no".
Dn_Ab 14y ago
It is a declarative language. But the underlying math of SQL is that of relations. It's closer to prolog than to Haskell. I don't know much about the theory of SQL but at least I can draw on the analogy of deriving algorithms in a Relational setting vs a functional to emphasize that the essence of functions is not the same as the essence of relations: Unlike functions, relations can be nondeterministic and each relation has a converse.
gaius 14y ago
In PL/SQL a "function" in the FP sense would be called "deterministic".

michaelochurch 14y ago

Functional programming has to contend with "the real world", which is stateful. FP isn't about eliminating computational effects, but managing them. The virtue of functional programming is that most of what happens is stateless, so that when stateful effects are needed, they're simple, uncoupled, and do exactly what one expects them to do. One principle for these sorts of applications is the "command/query separation". In essence, a procedure exposed at an interface level should either (a) do something, and do exactly what the user expects, or (b) answer a question about the system without computational effects. The case of (a) might be called "side effects", but that's a misnomer in this case because the requested state changes are the main effects. Functional programming is really about having the most basic "building block" of a program be a referentially-transparent function, and to handle stateful effects either using an abstraction such as an Action DSL (which represents a collection of stateful behaviors, which are computed statelessly and then executed or "committed" in a stateful environment, relegating the statefulness to one "commit" function) or a monadic representation. In contrast, imperative programming uses computational actions as the atomic "building block". For small programs, either imperative or functional programming will work. Imperative programming isn't evil; sometimes (but not often) it's the right approach. The problem emerges as programs get larger and more complicated, and more hands pass over the code. If a function is referentially-transparent, its semantics are usually tractable. If proper tests exist for it, optimizing the function (e.g. for performance) shouldn't affect the rest of the system. The constraint to which the function's maintainer is held is that the semantics shouldn't change, and any alteration that does change its semantics is an objective bug. The function might be memoized (that's a private "side effect" that doesn't affect referential transparency) but it still has to behave the same way. That's a really good property. In an imperative language where functions are replaced by computational actions, these actions are frequently made more complex and featureful as time goes on, and unexpected interactions emerge. Semantics can be defined for imperative programs, but doing so is more complicated and it's rarely done in real-world professional programming. Most clusterfuck awful systems in software, for the record, are complex systems with imprecise semantics. I would say that 90% of software failures I've seen can be characterized in this way. This sort of thing can happen in any language, but it seems to be a lot more common in imperative programming, especially in verbose languages like Java or C++. It's just much harder to enforce precise semantics over an anything-goes imperative style of programming than a functional one. As for databases... well, reality is that purely functional programming isn't always possible. We're not just computing mathematical functions; we're trying to do things in the physical world. We need persistence, and that means telling a robotic arm to write bytes on magnetic media (stateful effect). There are several approaches that one can use. One that I like (and it has a lot of desirable properties in distributed systems) is the append-only approach: update and delete operations simply aren't allowed, although new records can shadow old ones in the context of time-sensitive queries. For example, the temperature in New York is 56 degrees now (8:30 am, May 7). In six hours, it might be 70. That doesn't mean we throw the old record away. It's still valid: the temperature was 56 at 8:30. The answer to the query, "What was the temperature at 8:30 in New York?" should never change. On the other hand, the answer to the query, "What is the most recent temperature in New York?" will change all the time.