4 ms·
If you're interested in this topic, we're building a tool for building cluster manager components (like policy-based container schedulers) using constraint prog
by lsuresh 5y ago
If you're interested in this topic, we're building a tool for building cluster manager components (like policy-based container schedulers) using constraint programming, where constraints are specified using SQL.
Paper: https://www.usenix.org/system/files/osdi20-suresh.pdf https://www.usenix.org/system/files/osdi20-suresh.pdf
Code: https://github.com/vmware/declarative-cluster-management/ https://github.com/vmware/declarative-cluster-management/
- gavinray 5y agoHoly smokes, this is one of the coolest thing I've ever seen. This lets you use data from standard, JDBC-supported SQL databases and write constraints as plain SQL-like queries?! This reminds me of an idea I had about embedding a solver API in Postgres using it's extension functionality and being able to make SQL "VIEWS" that represent solver results, like a delivery route or employee roster. CREATE VIEW todays_employee_schedule AS SELECT ... CHECK row_satisfies_constraints(...) If I'm understanding this properly, this seems like this could be applied to general-purpose problems as well -- not just the one you're targeting right? // Create an in-memory database and get a JOOQ connection to it final DSLContext conn = DSL.using("jdbc:h2:mem:"); // A table representing some machines conn.execute("create table machines(id integer)"); // A table representing tasks, that need to be assigned to machines by DCM. // To do so, create a variable column (prefixed by controllable__). conn.execute("create table tasks(task_id integer, controllable__worker_id integer, " + "foreign key (controllable__worker_id) references machines(id))"); // Add four machines conn.execute("insert into machines values(1)"); ... // Add two tasks conn.execute("insert into tasks values(1, null)"); ... // Time to specify a constraint! Just for fun, let's assign tasks to machines such that the machine IDs sum up to 6. final String constraint = "create constraint example_constraint as " + "select * from tasks check sum(controllable__worker_id) = 6"; // Create a DCM model using the database connection and the above constraint final Model model = Model.build(conn, List.of(constraint)); // Solve and return the tasks table. The controllable__worker_id column will either be [1, 5] or [5, 1] final List<Integer> column = model.solve("TASKS").map(e -> e.get("CONTROLLABLE__WORKER_ID", Integer.class));
- lsuresh 5y agoThank you for the kind words! Yes it could be applied to general problems as well. In fact, we used it once to plan a program committee meeting. The library (DCM) has no idea what problem setting its being used for, and has no semantic info about the problem other than the schema and constraints we tell it about. That said, the current focus is on things like incremental computation, debuggability, automatically subsetting the relevant state from the DB, and scalability to really large cluster sizes (O(50K) nodes), which are more useful in the cluster management context than general-purpose constraint programming tasks. Edit: Also worth mentioning is that your intuition is spot on. The earlier versions of the library went with a CREATE VIEW syntax as you wrote it out. Now that we have a customizable parser, we have since changed it to CREATE CONSTRAINT for clarity.