5 ms·
Wow, I'm currently reading the Scallop paper, so funny to see it posted here! I really love the concept. This isn't just differentiable neurosymbolic declarati
by versteegen 2y ago
Wow, I'm currently reading the Scallop paper, so funny to see it posted here!
I really love the concept. This isn't just differentiable neurosymbolic declarative probabilistic programming; Scallop has the flexibility of letting you use various (18 included) or custom provenance semirings to e.g. track "proofs" why a relational fact holds, not just assign it a probability. Sounds cool but I'm still trying to figure out the practicality.
Also worth pointing out that it seems that a lot of serious engineering work has been done on Scallop. It has an interpreter and a JIT compiler down to Rust compiled and dynamically loaded as a Python module.
Because a Scallop program (can be) differentiable it can be used anywhere in an end-to-end learning system, it doesn't have to take input data from a NN and produce your final outputs, as in all the examples they give (as far as I can see). For example you probably could create a hybrid transformer which runs some Scallop code in an internal layer, reading/writing to the residual stream. A simpler/more realistic example is to compute features fed into a NN e.g. an agent's policy function.
The limitation of Scallop is that the programs themselves are human-coded, not learnt, although they can implement interpreters/evaluators (e.g. the example of evaluating expressions).
- alankarmisra 2y agoI'm wondering if this is a limitation though. If it can be learnt from training data, would it not be part of the neural network training data? I imagine we use Scallop to bridge the gap where we can't readily learn certain rules based on available data or perhaps we would prefer to enforce certain rules?
- daveguy 2y agoI'm pretty sure "differentiable" isn't necessary or sufficient to create valid and useful code.
- adastra22 2y agoOn the one hand, there are problems which by accident or design are nondifferentiable. Cryptography, for example. In the other hand, these problems are routinely analyzed and solved by differentiable algorithms running on neural net substrates (e.g. you).
- sitkack 2y agoPapers are linked here https://www.scallop-lang.org/resources.html https://www.scallop-lang.org/resources.html https://www.cis.upenn.edu/~mhnaik/papers/neurips21.pdf https://www.cis.upenn.edu/~mhnaik/papers/neurips21.pdf https://dl.acm.org/doi/10.1145/3591280 https://dl.acm.org/doi/10.1145/3591280 There is a 135 page book on Scallop https://www.cis.upenn.edu/~mhnaik/papers/fntpl24.pdf https://www.cis.upenn.edu/~mhnaik/papers/fntpl24.pdf