Spark SQL does exactly that: declarative graph dataflow computations + a multicore (also distributed) scheduler and executor, although it doesn't really scale d

Spark SQL does exactly that: declarative graph dataflow computations + a multicore (also distributed) scheduler and executor, although it doesn't really scale down that well to a single machine and the execution layer happens mostly on the JVM, even when using PySpark (with an ungodly mix of of serialized Python and JVM objects and code). Ibis (also by Wes McKinney) does the first part, but it offloads scheduling and execution to the underlying database you are using.