3 ms·
Kinda depends on how you define a transformer solving the problem. I feel like you could fine-tune a transformer in the style of https://www.ai21.com/blog/juras
by chessgecko 4y ago
Kinda depends on how you define a transformer solving the problem. I feel like you could fine-tune a transformer in the style of https://www.ai21.com/blog/jurassic-x-crossing-the-neuro-symbolic-chasm-with-the-mrkl-system https://www.ai21.com/blog/jurassic-x-crossing-the-neuro-symb... to produce steps to solve it, ie
"<initial prompt>" =>
1. [LLM generate] write a python program to return an integer >500 digits with two different prime factors with >250 digits and return all three
2. [execute code generated by 1 and return result]
ChatGPT did actually generate code that worked when I manually put in step 1, it'll be interesting to see what systems people chain together
- zora_goron 4y agoThis was the Python code outputted by ChatGPT when I pasted in the original problem: import random import sympy def generate_large_int(num_digits): return int("".join(str(random.randint(0,9)) for _ in range(num_digits))) # Generate the first large integer first_int = generate_large_int(500) # Generate two prime factors of the first integer while True: factor1 = sympy.randprime(10\*250, 10\*251) factor2 = sympy.randprime(10\*250, 10\*251) if first_int % factor1 == 0 and first_int % factor2 == 0: break print(f"{first_int},{factor1},{factor2}","STOP")
- chessgecko 4y agoYeah to get the working result I had to reword the problem a little. In my post I kinda assume you could fine tune a model to do that. It is a little cheap though.
- golol 4y agoBy a transformer I essentially mean a model (probably LLM) that operates on a context-window of tokens by appending a new token and then shifting. If I understand correctly the solution you give would be an example of what I mean by using prompt engineering to solve the problem. It is like building a meta model using a transformer.