3 ms·
The problem with scallop is that no one has ever shown a single program that wasn't easier to write without it. Their papers usually contain also no examples, a
by light_hue_1 2y ago
The problem with scallop is that no one has ever shown a single program that wasn't easier to write without it. Their papers usually contain also no examples, and the ones that do are a heck of a lot more complicated than asking the model to do it directly. The programming languages world let's them get away with this because they're excited to be relevant in the LLM universe. But I wouldn't accept any of this work if I was their reviewer; they need to write real programs and compare against baselines. Many of them. And show that their language provides any value.
Just look at the examples on their website. All 3 are lame and far easier without their language.
It's like publishing that you have a new high performance systems language and never including any benchmark. They would be rejected for that. Things just haven't caught up in the ML+PL world.
- mentalgear 2y agoI think you misunderstand what a neuro-symbolic programming language (like Scallop) is for. It's not about performance, but safety. Making safe decisions becomes exponentially more important as ML / agents evolve, to avoid "performant" but ultimately inefficient/dangerous/wasteful inferences.
- light_hue_1 2y agoThen show me programs that meaningfully improves safety. And compare them to baseline options to demonstrate this. None of these examples improve safety beyond a trivial check on the output which I can also do with a simple prompt.