4 ms·

This just sounds like prompt engineering? I think you have the right thinking in regards to LLMs needing more logic based foundations but it probably needs to b

by proc0 1y ago

This just sounds like prompt engineering? I think you have the right thinking in regards to LLMs needing more logic based foundations but it probably needs to be designed into the architecture itself. If you are enhancing the prompts with structure I think it will still be trying to autocomplete that structure instead of actually using logic based reasoning. I think there's probably another foundational technique like transformers that could be added such that it can encode logical structures that it can then use when it needs to reason, but unfortunately I cannot experiment or do any research on this as it would probably take months or years with no guarantee of success.

GENIXUS 1y ago
Thanks for the thoughtful reply — I agree that what I’m doing may look like an advanced form of prompt engineering, and in a sense, it probably is. I’m very new to this field, so I don’t yet have the knowledge or resources to touch the architecture itself. That’s why I’ve been experimenting at the input level — trying to see how far structure alone can constrain or guide model behavior without changing the model. You’re absolutely right that the model still tries to “autocomplete” within the structure, and not truly “reason” in a formal sense. But the interesting part for me was that even without touching internals, I could get the model to simulate something that looks like logic-based reasoning — repeatable, deterministic responses within a controlled structure. That said, I totally agree: long-term, we’ll need architectural support to make real logic possible. I appreciate your insight — if you ever revisit this kind of research, I’d love to learn from it.
- proc0 1y ago
  Right. And to clarify prompt engineering became a buzz word but I think there's something tangible there where we'll need to really get familiar with how models behave and optimize the inputs accordingly.

by proc0 1y ago

GENIXUS 1y ago

Thanks for the thoughtful reply — I agree that what I’m doing may look like an advanced form of prompt engineering, and in a sense, it probably is. I’m very new to this field, so I don’t yet have the knowledge or resources to touch the architecture itself. That’s why I’ve been experimenting at the input level — trying to see how far structure alone can constrain or guide model behavior without changing the model. You’re absolutely right that the model still tries to “autocomplete” within the structure, and not truly “reason” in a formal sense. But the interesting part for me was that even without touching internals, I could get the model to simulate something that looks like logic-based reasoning — repeatable, deterministic responses within a controlled structure. That said, I totally agree: long-term, we’ll need architectural support to make real logic possible. I appreciate your insight — if you ever revisit this kind of research, I’d love to learn from it.

proc0 1y ago
Right. And to clarify prompt engineering became a buzz word but I think there's something tangible there where we'll need to really get familiar with how models behave and optimize the inputs accordingly.