3 ms·
is it possible to solve NP-hard problems with transformers/LLM?
by transformi 3y ago
is it possible to solve NP-hard problems with transformers/LLM?
- deleted 3y ago[deleted]
- ironbound 3y agoyou'll be limited right away by the context lenght of 4096 tokens
- cardboardbach 3y agoOf course.
- shawntan 3y agoYou should check out the work referenced in the abstract, the most recent here: https://twitter.com/lambdaviking/status/1630581475425828864 https://twitter.com/lambdaviking/status/1630581475425828864 There are limitations for what a Transformer can compute if we do not allow for Chain-of-Thought type of output. By allowing the model to "show its work" allows it to effectively use the output as an "infinite tape". I'm simplifying but that's the basic gist of it. I'll shamelessly plug my blog post from a week ago for a simpler take on the matter: https://blog.wtf.sg/posts/2023-02-03-the-new-xor-problem/ https://blog.wtf.sg/posts/2023-02-03-the-new-xor-problem/
- intalentive 3y agoThey can't even learn parity checks. https://arxiv.org/abs/2207.02098 https://arxiv.org/abs/2207.02098