3 ms·

"AI heats the planet"... really? You mean marginally? I'll assume you're asking in good faith. Using NNs allows this project to stand on the shoulders of giant

"AI heats the planet"... really? You mean marginally? I'll assume you're asking in good faith. Using NNs allows this project to stand on the shoulders of giants: philosophically, mathematically, programmatically, but also I expect this to plug in to OSS LLMs, and leverage their knowledge, similarly to how a human child learns in a Pavlovian/intuitive response, and only later starts to learn to reason. Wrt inefficiency, training will be inefficient, but the programs can be extracted to CPU instructions / CUDA kernels during inference. Also, I'm interested in using straight through estimators in the forward pass of training, to do this conversion in training too. Cyc looks cool, but from my cursory glance, is it capable of learning, or is its knowledge graph largely hand coded? Neurallambda is at least as scalable as an RNN, both in data and compute utilization.

still_grokking 3y ago
> Wrt inefficiency, training will be inefficient That's the "heating the planet part" I was referring to. :-) > but the programs can be extracted to CPU instructions / CUDA kernels during inference This just makes my original question more pressing: What are the NNs good for if the result will be normal computer programs? (Just created with astronomical overhead!) > Cyc looks cool, but from my cursory glance, is it capable of learning, or is its knowledge graph largely hand coded? The whole point is that it can infer new knowledge from known facts through a logical reasoning process. This inference process was run since 40 years. The result is the most comprehensible "world knowledge" archive ever created. Of course this wouldn't be possible to create "by hand". And in contrast to NN hallucinations there is real logical reasoning behind, and everything is explainable. I still don't get how some "dreamed up" programs from your project are supposed to work. Formal reasoning and NNs don't go well with each other. (One could even say they're opposites). Imho it's "real reasoning" OR "dreamed up stuff". How "dreamed up stuff" could improve "real reasoning"? Especially as the "dreamed up stuff" won't be included in the end results anyway, where only the formal things remain. To what effect are the NNs included in your project? (I mean besides the effect that the HW and energy demands will go through the roof, ending up billion times higher than just doing some lambda calculus directly…) And yes, these are genuine questions. I just don't get it. It looks for me like "let's do things maximally inefficiently, but at least we can put a 'works with AI' rubber stamp on it"; which is maybe good to collect VC money, but else? What do I overlook here?
- eru 3y ago
  > This just makes my original question more pressing: What are the NNs good for if the result will be normal computer programs? (Just created with astronomical overhead!) You know how expensive it is to pay humans to write 'normal' computer programs? In terms of both dollars and CO2.

still_grokking 3y ago

> Wrt inefficiency, training will be inefficient That's the "heating the planet part" I was referring to. :-) > but the programs can be extracted to CPU instructions / CUDA kernels during inference This just makes my original question more pressing: What are the NNs good for if the result will be normal computer programs? (Just created with astronomical overhead!) > Cyc looks cool, but from my cursory glance, is it capable of learning, or is its knowledge graph largely hand coded? The whole point is that it can infer new knowledge from known facts through a logical reasoning process. This inference process was run since 40 years. The result is the most comprehensible "world knowledge" archive ever created. Of course this wouldn't be possible to create "by hand". And in contrast to NN hallucinations there is real logical reasoning behind, and everything is explainable. I still don't get how some "dreamed up" programs from your project are supposed to work. Formal reasoning and NNs don't go well with each other. (One could even say they're opposites). Imho it's "real reasoning" OR "dreamed up stuff". How "dreamed up stuff" could improve "real reasoning"? Especially as the "dreamed up stuff" won't be included in the end results anyway, where only the formal things remain. To what effect are the NNs included in your project? (I mean besides the effect that the HW and energy demands will go through the roof, ending up billion times higher than just doing some lambda calculus directly…) And yes, these are genuine questions. I just don't get it. It looks for me like "let's do things maximally inefficiently, but at least we can put a 'works with AI' rubber stamp on it"; which is maybe good to collect VC money, but else? What do I overlook here?

eru 3y ago
> This just makes my original question more pressing: What are the NNs good for if the result will be normal computer programs? (Just created with astronomical overhead!) You know how expensive it is to pay humans to write 'normal' computer programs? In terms of both dollars and CO2.