6 ms·

This one of the ideas we have been attempting to realise in Eve (eg http://www.lighttable.com/2014/07/18/imperative-thinking/ http://www.lighttable.com/2014/07/

by jamii 12y ago

This one of the ideas we have been attempting to realise in Eve (eg http://www.lighttable.com/2014/07/18/imperative-thinking/ http://www.lighttable.com/2014/07/18/imperative-thinking/). We separate the data model (expressed as relations) from the physical implementation of that model (one or more physical data structures per table that implement the queryable interface). The compiler is then responsible for deciding how to evaluate a given query, based on the available data-structures, indexes, hereustics and user hints. It seems radical in a programming context but it is a pretty standard and well proven separation in the database world. My favourite work along this line is https://infosys.uni-saarland.de/publications/DJ11.pdf https://infosys.uni-saarland.de/publications/DJ11.pdf which defines the data model to be an append-only list of tuples and treats the creation of all other data-structures, indexes, views etc as part of a single global optimisation problem.

eru 12y ago
Have you read the "Out of the tarpit" paper (http://shaffner.us/cs/papers/tarpit.pdf http://shaffner.us/cs/papers/tarpit.pdf)? It deals with using relations in programs as opposed to just in your databases. At Standard Chartered we regularly used relations as a perfectly cromulent data structure in our programs.
- jamii 12y ago
  Many times :) Some of our other big inspirations are Bloom (http://boom.cs.berkeley.edu/index.html http://boom.cs.berkeley.edu/index.html) and OctopusDB (https://infosys.uni-saarland.de/projects/octopusdb.php#section_publications https://infosys.uni-saarland.de/projects/octopusdb.php#secti...).
ntoshev 12y ago
I thought LightTable is an IDE, not rethinking programming :) Anyway, great thinking, I've actually come to a lot of the same conclusions. I'd suggest an inspiration: spreadsheets. Can you implement the separation of data model from indexes/optimizing physical model in an ad-hoc spreadsheet-like UI? When thinking "spreadsheets" you should think about not just Excel, but also Lotus Improv (which is a more structured spreadsheet, which is both a strength and a weakness).
- jamii 12y ago
  Light Table is an IDE that attempted to simplify programming by improving the tooling on top of existing languages. Unfortunately, too many of the problems we are trying to fix are baked in at the language level and there is only so much an editor can do. Eve is our attempt at rethinking how languages should work. The UI we are currently working on is very spreadsheet-like. We show tables as grids. Rules are built by dragging columns of data into a working space. We deliberately don't support adhoc layout in the program itself but we will probably add some kind of presentation tool that links various tables into a nicely laid out report.
seanmcdirmid 12y ago
Sounds like Japan's 5th generation computing project. Heavy efforts were made in the 80s/early 90s in the logic/relational programming direction but ultimately failed. I think these efforts will ultimately succeed only when they are combined with ML, but there is currently very little cross over between PL and ML (I wonder what Peter Norvig will say about this at his Splash keynote).
- ibdknox 12y ago
  What is the missing piece that ML brings to the party? I can certainly see lots of neat opportunities for ML tie in with the system we're building, but I can't think of anything that would end up relying on it.
  - seanmcdirmid 12y ago
    ML brings much better heuristics to the party. Basically, the dream of declarative programming with hand written generic decision and data scheduling procedures hasn't really panned out. All the arrows point to it continuing not to pan out very well, while ML has recently blasted beyond our wildest dreams.
siddboots 12y ago
Eve sounds exciting, and ambitious :) Any idea when we can we see some more details?
- ibdknox 12y ago
  We hope to do a big reveal early next year, but we'll probably be talking more about the technology between now and then. :)

by jamii 12y ago

eru 12y ago

Have you read the "Out of the tarpit" paper (http://shaffner.us/cs/papers/tarpit.pdf http://shaffner.us/cs/papers/tarpit.pdf)? It deals with using relations in programs as opposed to just in your databases. At Standard Chartered we regularly used relations as a perfectly cromulent data structure in our programs.

jamii 12y ago
Many times :) Some of our other big inspirations are Bloom (http://boom.cs.berkeley.edu/index.html http://boom.cs.berkeley.edu/index.html) and OctopusDB (https://infosys.uni-saarland.de/projects/octopusdb.php#section_publications https://infosys.uni-saarland.de/projects/octopusdb.php#secti...).

ntoshev 12y ago

I thought LightTable is an IDE, not rethinking programming :) Anyway, great thinking, I've actually come to a lot of the same conclusions. I'd suggest an inspiration: spreadsheets. Can you implement the separation of data model from indexes/optimizing physical model in an ad-hoc spreadsheet-like UI? When thinking "spreadsheets" you should think about not just Excel, but also Lotus Improv (which is a more structured spreadsheet, which is both a strength and a weakness).

jamii 12y ago
Light Table is an IDE that attempted to simplify programming by improving the tooling on top of existing languages. Unfortunately, too many of the problems we are trying to fix are baked in at the language level and there is only so much an editor can do. Eve is our attempt at rethinking how languages should work. The UI we are currently working on is very spreadsheet-like. We show tables as grids. Rules are built by dragging columns of data into a working space. We deliberately don't support adhoc layout in the program itself but we will probably add some kind of presentation tool that links various tables into a nicely laid out report.

seanmcdirmid 12y ago

Sounds like Japan's 5th generation computing project. Heavy efforts were made in the 80s/early 90s in the logic/relational programming direction but ultimately failed. I think these efforts will ultimately succeed only when they are combined with ML, but there is currently very little cross over between PL and ML (I wonder what Peter Norvig will say about this at his Splash keynote).

ibdknox 12y ago
What is the missing piece that ML brings to the party? I can certainly see lots of neat opportunities for ML tie in with the system we're building, but I can't think of anything that would end up relying on it.
- seanmcdirmid 12y ago
  ML brings much better heuristics to the party. Basically, the dream of declarative programming with hand written generic decision and data scheduling procedures hasn't really panned out. All the arrows point to it continuing not to pan out very well, while ML has recently blasted beyond our wildest dreams.

siddboots 12y ago

Eve sounds exciting, and ambitious :) Any idea when we can we see some more details?

ibdknox 12y ago
We hope to do a big reveal early next year, but we'll probably be talking more about the technology between now and then. :)