6 ms·

One of the more interesting parts (to me) about Eve wasn't the interactive programming or the literate programming, but that it included a database inside of th

by iamwil 9y ago

One of the more interesting parts (to me) about Eve wasn't the interactive programming or the literate programming, but that it included a database inside of the programming language. Most programming languages think of a database as a thing separate to it (I think Smalltalk is an exception). Hence, they needed language constructs to make it easier to talk to databases. As a result, Eve was inspired by the Bloom language being researched at Berkeley, which was in turn inspired by an old language called Datalog. In it, they take constraints of a declarative language (like SQL) and use it to make distributed computing a lot easier. "Our main data-structure was now a tree of tables. Rather than one big top-level function, we switched to a pipeline of functions. Each function pulled data out of the global store using a datalog query, ran some computation and wrote data back. Having less nesting reduced the impact of lexical scope and cursor passing. Using datalog allowed normalising the data store, avoiding all the issues that came from hierarchical models. At this point we realised we weren't building a functional language anymore. Most of the programs were just datalog queries on normalised tables with a little scalar computation in the middle. We were familiar with Bloom and realised that it fit our needs much better than the functional pidgin we had built so far - no lexical scoping, no data-structures, no explicit ordering. In late March we began work on a Bloom interpreter." http://incidentalcomplexity.com/2014/10/16/retrospective/ http://incidentalcomplexity.com/2014/10/16/retrospective/ Here's a talk by Peter Alvaro (doing research with Bloom) about it, and I had to watch it 4 to 6 times to really get everything out of it. He makes a shout out to Eve at the end (https://www.youtube.com/watch?v=R2Aa4PivG0g https://www.youtube.com/watch?v=R2Aa4PivG0g). The Bloom guys are the same guys that released the paper to Anna, the KV store just 11 days ago. (https://news.ycombinator.com/item?id=16551072 https://news.ycombinator.com/item?id=16551072)

nickpsecurity 9y ago
Another example is combining a database with LISP via AllegroCache: https://franz.com/products/allegrocache/ https://franz.com/products/allegrocache/ IBM took the idea further with AS/400 integrating one at OS-level for various apps to use that OS could manage for them.
scriptproof 9y ago
"Excited, we presented our prototype to a small number of non-programmers and sat back to watch the magic. To our horror, not a single one of them could figure out what the simple example program did or how it worked, nor could they produce any useful programs themselves." (From the article linked above). As a programmer, I had the same reaction, actually.
- iamwil 9y ago
  That was pretty early on. They iterated a couple of times thereafter.
  - ibdknox 9y ago
    yeah, that was a very early version that we never released publicly, but was effectively a cleaned up version "Aurora," which I presented at Strange Loop once upon a time ago.
- tree_of_item 9y ago
  That quote isn't in reference to the version described in the comment you're replying to.
munificent 9y ago
> Most programming languages think of a database as a thing separate to it (I think Smalltalk is an exception). MUMPS [1] has a DB baked in. kdb+ [2] is a marriage of K and a DB. [1]: https://en.wikipedia.org/wiki/MUMPS https://en.wikipedia.org/wiki/MUMPS [2]: https://en.wikipedia.org/wiki/Kdb%2B https://en.wikipedia.org/wiki/Kdb%2B
- WaxProlix 9y ago
  I could be wrong, but my understanding is that MUMPS/M seems more analogous to a Database with a built-in language than a language with a built-in database, although having said that, the distinction seems pretty fine to me. Is Company X SQL, which can have sprocs and such, also a language with a database baked in? As an aside: Does anyone (other than Epic Health Systems or whoever) use Mumps/M at any kind of scale? I hear that the issues with global namespacing can become really burdensome above even small thresholds.
  - ta34551122 9y ago
    Petroleum industry used to use Pick where I lived for some distribution software, but it seems it's main place is medical records software from what I've heard). It is a fine distinction I suppose, although the Pick stuff I've looked at was more than just stored procedures - TUI/GUI support for apps (mostly dumb terminal based but Cache probably has some GUI stuff in there now), overnight batch scripts and report creation etc. I haven't used MUMPS, but did a little bit of time with Pick and later on Cache. I believe Pick systems were often more than just the multi-valued database and absolutely horrendous language, in that they were often tightly integrated into the OS too - disk storage, terminal support etc.
  - Roboprog 9y ago
    The system that gathers data for California birth certificates and related [pre]natal health stats reporting is written in MUMPS/M/Cache. I’ve not worked with the code for that, though - we are in the process of replacing it. The fellow that originally created the system decades ago has been in “semi retirement” for quite a while, and it’s hard to find people that want to work on the platform. So, we are doing a replacement in (mostly) Java, of course. And I don’t even especially like Java, but it pays the bills in the Sacramento area. I suspect the bigger scaling issues with the codebase are “people-ware”, and thus the push (in most shops and projects) to go least common denominator on programming platform choice, sadly.
ibdknox 9y ago
I completely agree that the semantics are probably the most overall useful thing we presented in this version of Eve. Having the world just be views over a datastore that you don't have to manage in any meaningful way is hugely liberating coming from the way we program now. You don't have to worry so much about how things are "placed" - you can always access them wherever and however you want. Bloom gave us a nice way of thinking about how to deal with change in that context so that it's not just some giant pile of scary mutable state. Removing the fetching and placing of data is probably the biggest source of Eve's efficiency too. It turns out a lot of the code we write is nothing more than plumbing things into the correct location.
- iamwil 9y ago
  > I completely agree that the semantics [of views over a datastore] are probably the most overall useful thing presented in this version of Eve. Were you guys planning a next version of Eve with a different useful semantic? Or was this idea of having the world as views over a datastore the last overall useful idea that you guys ended up with?
  - ibdknox 9y ago
    There were a few directions we went hereafter. One was thinking of the world in terms of simulations, another was statemachine oriented. There are a few things that were never quite right with the set of semantics here: things that really were an ordered set of steps (far fewer things than you think, but still some) were pretty hard to express and the distinction between commit and bind caused some weird things to happen at the edges. The former could be fixed by directly integrating statemachines into the language. The latter remains a bit of an open problem. Some things you want to manually manage the lifetime of, others you want to be tied to their supports. The problem is that you often want to mix and match those freely, which is just a tough thing to reason about. Taking a stronger stance that Eve was still a bit too much like programming, and not as directly modeling as we meant for it to be, can take you to some interesting places too.
    - hammerandtongs 9y ago
oldmanhorton 9y ago
I believe a major programming environment that you are missing here is the SAP environment and the ABAP language, which (from my minimal experience) is entirely database-centric and probably powers most of the large enterprise HR and commerce systems in the world.
dopamean 9y ago
I've been writing Elixir at work for the last 5 months and learned in that time that Elixir has DBs built in as well. ETS is a key value store and Mnesia (lol) is a bit more like a traditional db (I haven't actually used Mnesia yet). From what I understand these things are actually a part of OTP and so they ship with the whole environment.
- moomin 9y ago
  I seem to recall aphyr said he hadn’t bothered to Jensen mnesia because it was obviously so bad.
  - dopamean 9y ago
    ok
davidw 9y ago
I never had much to do with them, but stuff like dbase III and FoxPro and the like were integrated DB/language environments, and they were quite popular back in the day.
- stareatgoats 9y ago
  Mmm... Ashton-Tate dBase III+, those were the good old days:) And later, similarly with db and programming integration: MS Access, and even Excel to some extent. None of them ever made it past the 'desktop' as a whole though, probably a feature of any db-included programming language.
  - Roboprog 9y ago
    I did quite a bit of dBase/Clipper back in the late 80s. The resulting code was much cleaner than most of the “enterprise” crap I have to deal with now, but xBase had some serious shortcomings: Corrupted indices in a multi-user (LAN) environment No roll-back (since no transactions) No referential integrity constraints Yeah, I sometimes miss it, but I understand why it died.
    - stareatgoats 9y ago
      I hear you. Yea, I never got round to using Clipper but was on track, especially the possibility to create standalone applications as front-end clients in a client/server setup was interesting. It was bought and botched by CA before I got round to it though, at least it looked like CA mishandled it from what I could see. A fun related fact albeit OT was that those DOS applications were orders or magnitude faster to keyboard operate than the GUI applications that came after ... I'm sure some of them are still in use for that reason.
cmeiklejohn 9y ago
The Lasp programming system also has an eventually consistent key-value store built-in (and, we've scaled it to 1,024+ nodes). Project home: https://lasp-lang.readme.io/docs https://lasp-lang.readme.io/docs Results on scaling: https://arxiv.org/abs/1708.06423 https://arxiv.org/abs/1708.06423
SolarNet 9y ago
The whole thing actually reminds me of Inform7, which is a natural language implementation of text adventures on a similar (albeit very old) database query language system.
- Zanni 9y ago
  I just wrote a tiny parser game in Inform, and the language is fascinating. You can create complex, fully interactive scenes in just a few sentences. For example, with just "There is a billiards table in the Gazebo. On it is a trophy cup. A starter pistol is in the cup." Inform would create three objects (a billiards table, a trophy cup and a starter pistol) in a previously established room (the Gazebo), and would deduce that the billiards table is a supporter (you can put things on it) and the trophy cup is a container (you can put things in it). With no further code you can examine all three items, remove the pistol from the cup and put it on the table, and so on.
eggy 9y ago
Picolisp has an integrate database [1] which the author has been using to build applications for a living since the 80s or 90s (I cannot recall when). [1] https://picolisp.com/wiki/?home

by iamwil 9y ago

nickpsecurity 9y ago

Another example is combining a database with LISP via AllegroCache: https://franz.com/products/allegrocache/ https://franz.com/products/allegrocache/ IBM took the idea further with AS/400 integrating one at OS-level for various apps to use that OS could manage for them.

scriptproof 9y ago

"Excited, we presented our prototype to a small number of non-programmers and sat back to watch the magic. To our horror, not a single one of them could figure out what the simple example program did or how it worked, nor could they produce any useful programs themselves." (From the article linked above). As a programmer, I had the same reaction, actually.

iamwil 9y ago
That was pretty early on. They iterated a couple of times thereafter.
- ibdknox 9y ago
  yeah, that was a very early version that we never released publicly, but was effectively a cleaned up version "Aurora," which I presented at Strange Loop once upon a time ago.
tree_of_item 9y ago
That quote isn't in reference to the version described in the comment you're replying to.

munificent 9y ago

> Most programming languages think of a database as a thing separate to it (I think Smalltalk is an exception). MUMPS [1] has a DB baked in. kdb+ [2] is a marriage of K and a DB. [1]: https://en.wikipedia.org/wiki/MUMPS https://en.wikipedia.org/wiki/MUMPS [2]: https://en.wikipedia.org/wiki/Kdb%2B https://en.wikipedia.org/wiki/Kdb%2B

WaxProlix 9y ago
I could be wrong, but my understanding is that MUMPS/M seems more analogous to a Database with a built-in language than a language with a built-in database, although having said that, the distinction seems pretty fine to me. Is Company X SQL, which can have sprocs and such, also a language with a database baked in? As an aside: Does anyone (other than Epic Health Systems or whoever) use Mumps/M at any kind of scale? I hear that the issues with global namespacing can become really burdensome above even small thresholds.
- ta34551122 9y ago
  Petroleum industry used to use Pick where I lived for some distribution software, but it seems it's main place is medical records software from what I've heard). It is a fine distinction I suppose, although the Pick stuff I've looked at was more than just stored procedures - TUI/GUI support for apps (mostly dumb terminal based but Cache probably has some GUI stuff in there now), overnight batch scripts and report creation etc. I haven't used MUMPS, but did a little bit of time with Pick and later on Cache. I believe Pick systems were often more than just the multi-valued database and absolutely horrendous language, in that they were often tightly integrated into the OS too - disk storage, terminal support etc.
- Roboprog 9y ago
  The system that gathers data for California birth certificates and related [pre]natal health stats reporting is written in MUMPS/M/Cache. I’ve not worked with the code for that, though - we are in the process of replacing it. The fellow that originally created the system decades ago has been in “semi retirement” for quite a while, and it’s hard to find people that want to work on the platform. So, we are doing a replacement in (mostly) Java, of course. And I don’t even especially like Java, but it pays the bills in the Sacramento area. I suspect the bigger scaling issues with the codebase are “people-ware”, and thus the push (in most shops and projects) to go least common denominator on programming platform choice, sadly.

ibdknox 9y ago

I completely agree that the semantics are probably the most overall useful thing we presented in this version of Eve. Having the world just be views over a datastore that you don't have to manage in any meaningful way is hugely liberating coming from the way we program now. You don't have to worry so much about how things are "placed" - you can always access them wherever and however you want. Bloom gave us a nice way of thinking about how to deal with change in that context so that it's not just some giant pile of scary mutable state. Removing the fetching and placing of data is probably the biggest source of Eve's efficiency too. It turns out a lot of the code we write is nothing more than plumbing things into the correct location.

iamwil 9y ago
> I completely agree that the semantics [of views over a datastore] are probably the most overall useful thing presented in this version of Eve. Were you guys planning a next version of Eve with a different useful semantic? Or was this idea of having the world as views over a datastore the last overall useful idea that you guys ended up with?
- ibdknox 9y ago
  There were a few directions we went hereafter. One was thinking of the world in terms of simulations, another was statemachine oriented. There are a few things that were never quite right with the set of semantics here: things that really were an ordered set of steps (far fewer things than you think, but still some) were pretty hard to express and the distinction between commit and bind caused some weird things to happen at the edges. The former could be fixed by directly integrating statemachines into the language. The latter remains a bit of an open problem. Some things you want to manually manage the lifetime of, others you want to be tied to their supports. The problem is that you often want to mix and match those freely, which is just a tough thing to reason about. Taking a stronger stance that Eve was still a bit too much like programming, and not as directly modeling as we meant for it to be, can take you to some interesting places too.
  - hammerandtongs 9y ago

oldmanhorton 9y ago

I believe a major programming environment that you are missing here is the SAP environment and the ABAP language, which (from my minimal experience) is entirely database-centric and probably powers most of the large enterprise HR and commerce systems in the world.

dopamean 9y ago

I've been writing Elixir at work for the last 5 months and learned in that time that Elixir has DBs built in as well. ETS is a key value store and Mnesia (lol) is a bit more like a traditional db (I haven't actually used Mnesia yet). From what I understand these things are actually a part of OTP and so they ship with the whole environment.

moomin 9y ago
I seem to recall aphyr said he hadn’t bothered to Jensen mnesia because it was obviously so bad.
- dopamean 9y ago
  ok

davidw 9y ago

I never had much to do with them, but stuff like dbase III and FoxPro and the like were integrated DB/language environments, and they were quite popular back in the day.

stareatgoats 9y ago
Mmm... Ashton-Tate dBase III+, those were the good old days:) And later, similarly with db and programming integration: MS Access, and even Excel to some extent. None of them ever made it past the 'desktop' as a whole though, probably a feature of any db-included programming language.
- Roboprog 9y ago
  I did quite a bit of dBase/Clipper back in the late 80s. The resulting code was much cleaner than most of the “enterprise” crap I have to deal with now, but xBase had some serious shortcomings: Corrupted indices in a multi-user (LAN) environment No roll-back (since no transactions) No referential integrity constraints Yeah, I sometimes miss it, but I understand why it died.
  - stareatgoats 9y ago
    I hear you. Yea, I never got round to using Clipper but was on track, especially the possibility to create standalone applications as front-end clients in a client/server setup was interesting. It was bought and botched by CA before I got round to it though, at least it looked like CA mishandled it from what I could see. A fun related fact albeit OT was that those DOS applications were orders or magnitude faster to keyboard operate than the GUI applications that came after ... I'm sure some of them are still in use for that reason.

cmeiklejohn 9y ago

The Lasp programming system also has an eventually consistent key-value store built-in (and, we've scaled it to 1,024+ nodes). Project home: https://lasp-lang.readme.io/docs https://lasp-lang.readme.io/docs Results on scaling: https://arxiv.org/abs/1708.06423 https://arxiv.org/abs/1708.06423

SolarNet 9y ago

The whole thing actually reminds me of Inform7, which is a natural language implementation of text adventures on a similar (albeit very old) database query language system.

Zanni 9y ago
I just wrote a tiny parser game in Inform, and the language is fascinating. You can create complex, fully interactive scenes in just a few sentences. For example, with just "There is a billiards table in the Gazebo. On it is a trophy cup. A starter pistol is in the cup." Inform would create three objects (a billiards table, a trophy cup and a starter pistol) in a previously established room (the Gazebo), and would deduce that the billiards table is a supporter (you can put things on it) and the trophy cup is a container (you can put things in it). With no further code you can examine all three items, remove the pistol from the cup and put it on the table, and so on.

eggy 9y ago

Picolisp has an integrate database [1] which the author has been using to build applications for a living since the 80s or 90s (I cannot recall when). [1] https://picolisp.com/wiki/?home