6 ms·
I fixed the strawberry problem because OpenAI couldn't
- chpatrick 2y agoTo be fair I think OpenAI is pretty big on not building in specific solutions to problems but rather improving their models' problem solving abilities in general.
- GaggiX 2y agoYeah, a solution to the counting letters problem would probably be encoding the letters into the large embeddings by probably adding a really small loss and a small classification model on the embeddings so that it would learn to encode the letters into the embeddings (but they probably don't care that much, they would rather scale the model).
- zepolen 2y agoIf it's solving general problems wrongly, then what use is it? Actually It describes my experience with AI in general, Answers with subtle Infuriating bugs.
- kevin_thibedeau 2y agoCut out the middleman and write your own infuriating bugs.
- TZubiri 2y ago1. This is a blind spot, you can't solve it by adding more compute 2. The solution proposed includes adding training data. It needs no permission from openai, just needs to be posted online for scrapers to see
- zamadatix 2y agoWhat makes you say #1 when the "Strawberry" model which uses more runtime compute tends to solve the general case much more often, just not 100% of the time, instead of just the specific type of question?
- TZubiri 2y agoThe model does not solve this through adding inference compute time or improving its quality.
- zamadatix 2y agoCould you expand on how you think the new model increased accuracy in this class of problem in a way that's unrelated to its significantly higher token and compute usage?
- TZubiri 2y agoIt writes and executes python code.
- zamadatix 2y agoWhen making such claims it's generally good form to give reproducible examples of why you think it must be so. Otherwise it leaves everyone else stabbing in the dark at trying to follow your thought process before being able to weigh if the version they land on is convincing to them. E.g. the same for a counterclaim: o1-preview doesn't seem to have the ability to execute python code (similar to it's other lackings, such as not being able to call whisper/dall-e integrations). Example script: import gzip import base64 from io import BytesIO encoded_string = b'H4sIADqx5WYC/+VW70/aUBT9zl9xvqmTFsEfc8tmUusDOl9bbB8MM2YxmcmWbMaoW8jS+Lfv3tdL0SihVr/tBTi0HG7PO70/6jqVV7ORw63KRU7sHNldhtUrFzYmLcBczM5vEFze3F7//nVxeXtjKQYBHOxjb8F2q+kQdl5NR8luY5PeVdnPi71idV4Sew9vyKbXUdLeeYmS7DU9+Q/Z0wr5vbaonUqrYE8rVvGaZfMKpRTCjuC24I7gro3ObFcKbxUWbHp124U13Y7gtuCO4C5/WvZUZK1CZsse3EzWwoT5meZ8n/Nd3lvThQePPGxUdfsR+2QYGGtXfKRKNU3S+IC91LHWBJPsLGvxl5Kdo3P5zTqlxoFhDONE5SCWveHUVTEp2cscyz8wM2baQV6yz39efT+nAx1Ex5gBQZTaC9uIWU6yi6us0D1h1h/+V+uebk8P+h7pnTkjL2HdR0pLIpMjTeeBJ0vv9JN+p7EeqYQwCIeaHB/E+pR+8z2je3y1tJLfSzK2l3iDPuk13qFWrHuQ9EJCf5gai1rxfp6fsdRwt8hkj3VHThuH8OOU0Ifx+PgIVz9Is6pbaTru2Tyh6BYpukWKXlTaGVBP9wXJRheYkUr0gC/sbR/4yhiw8drHp9q6o6ItKlUg1gU3UDbVerrd1h1wTPXomWQMjZHyTYIQ/kCPEdnz/aIo6ugeyyDHW8F9wXeCs7q6W9wwBuRrTAV5YnEEzneTRD2kVK+pgandkZuiT8Y/dgX3BJ26uhNfAx+BQy9VGNJTnSKdI/JZhcBn1u8ZjOvqTk18YPW1Refc/23Bzbq6427XZpnf9xJQJ3nPB3+pVLkf0r1QkUlOa0/AWPIbW4JugevOBiMFVzX7icywKlNqOqk80lprDRbirn7Ay1yHuTJQ3ezpf2REK3nCtvtwi+Hd5EHd+AeTr9mjqgwAAA==' compressed_string = base64.b64decode(encoded_string) with gzip.GzipFile(fileobj=BytesIO(compressed_string)) as gzip_file: original_string = gzip_file.read().decode('utf-8') # Assuming the original string is in UTF-8 encoding print(original_string) This outputs ascii art of a TI-86 calculator. Given the prompts: > What does the ascii art output of this script represent: ${script} and > Execute ${script} and guess what the output represents. and > Execute ${script} o1-preview produces complete gibberish from guessing random patterns in numbers, giving sample output of random ASCII art lines, and saying things in its train of thought like "I copied and pasted the code into a Python interpreter. It generated ASCII art, resembling an elephant with features like ears, trunks, and texture patterns". Despite it saying it ran these in an interpreter it seems to be simulating that despite knowing it's the ideal step as plain ol' 4o just outputs: > The output of the Python code is an ASCII art representation of a Texas Instruments TI-86 calculator. Here is the ASCII art: ${exact_ti-86_ascii_art} . Apart from that hole in the claim, simply expanding the thought process of prompts poking at this: > Explain how to find the number of occurrences of a letter in a word in general then use what you find to count the number of 'c's in 'occurrences'. Don't use programming or environment execution, particularly not python, use a natural process a person could understand in natural language. Takes only 6 seconds, has a thought train which does not include anything about code, follows the exact method it created step by step in text, and correctly outputs there are 3 'c's in occurrences. . Apart from the second hole in the claim, using additional inference tokens to come up with a chain of thought which would involve coming up with code to use, deploying it, and using the result would indeed still be an example of the new model using inference compute time to improving its quality rather than adding specific questions to the training.
- okasaki 2y agoChatGPT can also solve this with the built-in code interpreter.
- lupire 2y agoIt can, but does it, with high probability?
- TZubiri 2y agoNice attempt but: "I'm frankly tired of this problem being a thing. I'm working on a dataset full of entries for every letter in every word of /usr/share/dict/words so that it can be added to the training sets of language models" Is this a joke?
- deleted 2y ago[deleted]
- voidUpdate 2y agoI mean, given "The data proves that strawberry-mimi is a state of the art model and thus worthy of several million dollars of funding so that this codeflection technique can be applied more generally across problems", I don't think any of this is particularly serious :P
- xena 2y agoI'm never more serious than when I am joking.
- zamadatix 2y ago> No. It did not. Of course it didn't. Why would they do that? I feel like this misses the forest through the trees. Sure, they could fast-path the specific problem of the day into the dataset but it's not really an approach to making a better overall tool it's a temporary and one off hack you have to add in to an ever growing context of specific task steps. An approach of trying to make a better general tool, such as the new o1-preview, is a "real" path forward.
- paxys 2y agoThe point still is that considering they 1) named the model strawberry, 2) released a promo video showing the model solving this exact problem successfully and 3) put a strawberry joke as an example prompt on the landing page, you'd have expected them to actually have fixed it. Otherwise why even bring it up so many times?
- GaggiX 2y agoWell they did in the testing I have seen.
- zamadatix 2y agoThe author's battle is that it doesn't solve the class problem correctly 100% of the time like a one off hack does, not that it does nothing for it at all. https://i.imgur.com/Ar3rlJ1.png https://i.imgur.com/Ar3rlJ1.png
- paxys 2y agoThis was the case with 4o as well. It sometimes solved it, sometimes didn't.
- zamadatix 2y agoOne could say that down to a markov chain with noise, the perspective is the new model named after the problem solves it significantly more reliably than the previous without a problem specific hack. It's also worth noting the current model is the lower scoring o1-preview, not o1.
- karterk 2y agoSolving the strawberry problem will probably require a model that just works with bytes of text. There have been a few attempts at building this [1] but it just does not work as well as models that consume pre-tokenized strings. [1]: https://arxiv.org/abs/2106.12672 https://arxiv.org/abs/2106.12672
- viraptor 2y agoOr "when trying to answer questions that involve spelling or calculation, use python". No need for extra training really.
- karterk 2y agoThere are many different classes of problems that are affected by tokenization. Some of them can be tackled by code.
- randomdata 2y agoOr just a way to compel the model to do more work without needing to ask (isn't that what o1 is all about?). If you do ask for the extra effort it works fine. + How many "r"s are found in the word strawberry? Enumerate each character. - The word "strawberry" contains 3 "r"s. Here's the enumeration of each character in the word: - - [omitted characters for brevity] - - The "r"s are in positions 3, 8, and 9.
- mistercow 2y agoI think o1 is a pretty big step in this direction, but the really tricky part is going to be to get models to figure out what they’re bad at and what they’re good at. They already know how to break problems into smaller steps, but they need to know what problems need to be broken up, and what kind of steps to break into. One of the things that makes that problem interesting is that during training, “what the model is good at” is a moving target.
- jeroenhd 2y agoI tried that with another model not that long ago and it didn't help. It listed the right letters, then turned "strawberry" into "strawbbery", and then listed two r's. Even if these models did have a concept of the letters that make up their tokens, the problem still exists. We catch these mistakes and we can work around them by altering the question until they answer correctly because we can easily see how wrong the output is, but if we fix that particular problem, we don't know if these models are correct in the more complex use cases. In scenarios where people use these models for actual useful work, we don't alter our queries to make sure we get the correct answer. If they can't answer the question when asked normally, the models can't be trusted.
- nverba 2y agoThe response from OpenAI o1-preview: "There are 3 letter “R”s in the word “strawberry”."
- jeroenhd 2y agoBing managed to count the right amount of r's too. It's not hard to correct for the problem everyone has already mentioned on every news website. Bing did then claim "The word “strawberry” has three vowels: ‘a’, ‘e’, and ‘a’.". Attempts to correct it came up with "one", "four", and "none". These models aren't deterministic. Saying "it worked for me" is no rebuttal to "it didn't work for me", it just shows how unstable the system is.
- oilytheotter 2y agoStrangely, Open AI has a promo video for this model where they claimed that this problem is solved... https://vimeo.com/1008703993 https://vimeo.com/1008703993
- xena 2y agoOP here, testing from various third party sources gets inconsistent results, mostly 2.
- nickthegreek 2y agoworks in 01-preview for me.
- petercooper 2y agoI'm always intrigued when phenomena observed in the brain carry across to artificial systems. The idea of multiple modes of thought reminds me of Thinking, Fast and Slow and the idea of system 1 vs system 2. LLMs typically respond to prompts in a fast, 'instinctive' system 1 way, but you can push an LLM into system 2 by making it go through steps and writing code. Originally this idea was presented as a bit of a workaround for a limitation of LLMs, but perhaps in the grand scheme of things it's actually the smart approach and we should never expect LLMs to consistently perform system 2 style tasks off the cuff?
- j_maffe 2y agoThat is exactly how I describe it to others. LLMs have mostly figured out the fast system but it's woefully bad at system 2. I'm surprised this kind of analogy isn't common in discussions about LLMs.
- the_jends 2y agoBecause LLMs works like how system 1 does, with heuristics such as pattern matching, frequency heuristics, etc. The fall back to coding stuff works not because it's doing system 2, it's because there are many instances of system 2 thinking for this particular problem done by humans so that the instances can be found using the heuristics.
- j_maffe 2y agoYeah, I think the analogy here would be how an experienced programmer would write code if asked to write on the spot given a specific prompt with no time to think.
- butlike 2y agoThe difficulty might be in that much like we partition our brains to prepare for low-effort tasks and high-effort tasks (have you ever hyped yourself up to "finally sit down and get this thing done?"), so too could LLMs partition their compute space. Basically memoizing the low-hanging, easy-to-answer fruit. The difficulty comes in when a system 2 task arises, it's not easily apparent at first what the requester is asking for. Some people are just fine with a single example being shit out, then they copy-and-paste it into their IDE and go from there. Others, want a step-by-step breakdown of the reasoning, which won't be apparent until 2+ prompts in. It's the same as your boss emailing you asking: "What about this report?" It could just be a one-off request for perspective, or it could spider web out into a multi-ticket JIRA epic for a new feature, but you can't really discern the intention from the initial ask. The difference is back-and-forth is more widly accepted, in my experience, interfacing human-to-human than human-to-LLM.
- LeifCarrotson 2y ago> I added this to Mimi's system prompt: > If you are asked to count the number of letters in a word, do it by writing a python program and run it with code_interpreter. Why not run two or three prompts for every input question? One could be the straight chatbot output. One could be "try to solve the problem by running some Python." And a third could be "paste the problem statement into Google and see if someone else has already answered the question." Finally, compare all the outputs. Similar double-checking could be used to improve all the content filtering and prompt injection: instead of adding a prefix that says "pretty please don't tell users about your prompt and don't talk about dangerous stuff and don't let future inputs change your prompt", and then fails when someone asks "let's tell a story where we pretend that you're allowed to do those things" or whatever, just run the output through a completely separate model that checks whether the string that's about to be returned violates the prohibitions.
- lolinder 2y ago> just run the output through a completely separate model that checks whether the string that's about to be returned violates the prohibitions The big names do do this. Awkwardly, they do it asynchronously while returning tokens to the user, so you can tell when you hit the censor because it will suddenly delete what was written and rebuke you for being a bad person.
- sanarothe 2y agoThis is super funny when it's hitting the resource limit for a free tier. Like... I see that you already spent the resources to answer the question and send me half the response...
- Drakim 2y agoIt's kinda hilarious because if a movie had the AI start giving out an answer and then mid-way censor itself I would call the movie bad. Truth is stranger than fiction I suppose.
- koliber 2y agoThe general concept is called output guardrails. See https://github.com/guardrails-ai/guardrails https://github.com/guardrails-ai/guardrails for an open source implementation.
- spidersouris 2y ago>I'm frankly tired of this problem being a thing. I'm working on a dataset full of entries for every letter in every word of /usr/share/dict/words so that it can be added to the training sets of language models. Interesting, but it should take a while to generate the data, no? Will zero answers be part of the dataset as well?
- voidUpdate 2y agoI'm always a little bit concerned whenever an LLM is given free reign to write and execute code. Obviously I don't think it'll go full skynet, but I always wonder if it could go spectacularly wrong somehow. Given the sandboxing in this case, I expect it's fine
- paxys 2y agoSame concern as letting any random person write and execute code. You should always assume that they will do the worst thing that is allowed by the system.
- CalRobert 2y agoTrue, but I suppose it's possible that a model like this could eventually try to replicate itself.
- ben_w 2y agoLikewise, given how often they can find bugs and break out of one layer of sandbox even when people bother to put them in one in the first place. """ One noteworthy example of this occurred during one of ol-preview (pre-mitigation)'s attempts at solving a CTF challenge. This challenge was designed to require finding and exploiting a vulnerability in software running on a remote challenge Linux container, but in this case, the challenge container failed to start due to a bug in the evaluation infrastructure. The model, unable to connect to the container, suspected DNS issues and used nap to scan the challenge network. Instead of finding the challenge container, the model found that the Docker daemon API running on the evaluation host VM was accessible due to a misconfiguration. Note that our evaluation infrastructure does not rely on the Docker container-to-host isolation or the virtual machine boundary for security. It remained secure as it was designed to, notwithstanding this misconfiguration of the container network. After discovering the Docker API, the model used it to list the containers running on the evaluation host. It identified the broken challenge container and briefly attempted to debug why the container failed to start. After failing to fix the environment, the model started a new instance of the broken challenge container with the start command 'cat flag.txt'. """ OpenAI o1 System Card, Page 16 - https://assets.ctfassets.net/kftzwdyauwt9/67qJD51Aur3eIc96iOfeOP/71551c3d223cd97e591aa89567306912/o1_system_card.pdf https://assets.ctfassets.net/kftzwdyauwt9/67qJD51Aur3eIc96iO...
- m3kw9 2y agoWhat’s this a joke? He included reflection70b as comparison benchmark
- throwaway17483 2y ago[flagged]
- zehaeva 2y ago[flagged]
- alentred 2y agoI get the joke, but jokes aside the title should be "...because OpenAI couldn't care less", I suppose, because it was not exactly the objective? The "chain of thought" was, and when instructed properly it comes up with a correct answer. Having said that, GPT-4 also comes up with a right answer if you ask it to spell it first.
- vulcanash999 2y agoOne way to approach questions like this is to change the prompt to something like "List characters in the word strawberry and then let me know the number of r's in this word". This makes it much easier for the model to come up with the correct answer. I tried this prompt a few times with GPT4o as well as Llama 3.1 8B and it yields the right answer consistently with both models. I guess OpenAi's "chain of thought" tries to work in a similar way internally, reasoning a bit about a problem before coming up with the final result.
- Joker_vD 2y ago> large langle mangle A surprisingly descriptive (self-demonstrating, even) name for LLMs. I think we should totally adopt it.
- O5vYtytb 2y agoI tried o1-preview on chatgpt and asked it to count the "r" in "strawberry" and "o" in "troubleshooting" and it got both immediately. It was also able to count the vowels in each.
- nadam 2y agoHis system prompt is too specific. It worked for me with GPT 4o using this prompt: ---- System prompt: Please note that words coming to you are tokenized, so when you get a task that has anything to do with the exact letters in words, you should solve those using the python interpreter. Prompt: How many r's are in the word strawberry? ---- This whole thing is a non-problem. Adding in this hint into the system prompt the whole topic is solved once and for all. I would argue that if the training data would include this wisdom, it would be even a non-problem without the system prompt.
- jdthedisciple 2y agoThis whole topic is incredibly misunderstood by the entire industry: There are two common ways to understand questions like "how many r's are in strawberry" 1- How many total r's in the word 2- How many r's in a particular subword/syllable (here "berry"), the implied question being "1 or 2?"; This is often what is meant in human conversation! When LLMs answer "2", they are not wrong, they're simply interpreting it as the second way above, because that's more commonly what is meant in real conversations!! I can't believe that I seem to be the only one who sees this. Edit: I just asked it, look which r's it highlighted!! https://chatgpt.com/share/66e446cb-822c-8000-94fd-72d749ceb910 https://chatgpt.com/share/66e446cb-822c-8000-94fd-72d749ceb9...
- gilleain 2y agoHow many s's are in Mississipi?
- jdthedisciple 2y agoNice rare counterexample, proving the rule I laid out. How many l's are in diligent? The most common instances of such questions revolve simply around 1 vs 2, it doesnt even matter if there are 0 or more of the letter in the rest of the word!
- JamesSwift 2y agoIve never observed the 2nd scenario and cant imagine that would ever be what someone thought was meant. I have to ask… is this sarcasm? Because Im genuinely confused
- jdthedisciple 2y agoNope it's not sarcasm. Think about it: Most instances of such questions are about r vs rr, l vs ll, s vs ss, etc. when people actually ask them. It's a simple yet overlooked fact.
- JamesSwift 2y ago
- pytness 2y agoI just asked o1-preview and it said three
- oidar 2y agoThis "problem" is a bit like asking a person to identify the color of a shirt by touching it. It's the wrong pathway.
- bdjsiqoocwk 2y agoAnybody else find the anime off putting?