11 ms·
GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text
- extasia 3y ago>It is counter-intuitive that LLMs can exhibit such resilience despite severe disruption to input tokenization caused by scrambled text. I'm not sure that i agree. an LLM maximising the likelihood of its output could surely permute its input in such a way that it unscrambles the text? Need to read a little deeper and will report back. edit: interesting result, but the paper doesn't present a good reason that this would be "counter-intuitive" imo.
- spuz 3y agoOne way to consider how an LLM "sees" text is to imagine a character based language like Chinese - each symbol is a syllable which can be a word on its own or part of a word. If you scramble words at the character level, you are going to produce combinations of letters that don't match any known symbol. It would be like drawing a series of random strokes and asking someone who knows Chinese what it means. If you look at the example given in the paper, the word "won" is a single token. When it is scrambled as "wno" it is tokenised as "w" and "no" both of which are unrelated to the original token "won". Somehow the LLM is able to relate these two completely different tokens "w" and "no" back to the original token "won". I think the paper is claiming this is surprising because these tokens shouldn't have any correlation with each other in its training data.
- thaumasiotes 3y ago> If you scramble words at the character level, you are going to produce combinations of letters that don't match any known symbol. It would be like drawing a series of random strokes and asking someone who knows Chinese what it means. You can see an example of doing that very thing here: https://pbs.twimg.com/media/EG1ZV_tX4AALiIG?format=jpg&name=large https://pbs.twimg.com/media/EG1ZV_tX4AALiIG?format=jpg&name=... . There is a hashtag at the bottom of the image explaining the meaning of the nonexistent character, but if you remove that from the image, people understand it just as quickly.
- evertedsphere 3y agoPartial counterpoint: if you were to mutate characters (by changing less distinctive components to other vaguely similar-looking ones) or to reorder the order of characters in hanzi/kanji compounds, I would imagine that a native speaker would still be able to read them with some difficulty. I can attempt to produce a Japanese example by going to town on an example from Jreibun, but note that I am far from native: 後食に罠くなるのは生里像現なので壁けることはできないが、午後の事士の校率が干がるので木っている。 As far as I'm concerned, swapping out the radicals doesn't hurt that much (this is usually a negative, since it leads you to confuse character pairs like 候 and 侯, especially if you don't practice writing) and swapping the order of characters is a bit more annoying. That said, a Mandarin one would be more convincing, since reordering the various markers that serve the roles of Japanese verb conjugations would be less disruptive than turning できなかった into っぎかてなた, which I did not do for that reason. -- (The original sentence was 食後に眠くなるのは生理現象なので避けることはできないが、午後の仕事の効率が下がるので困っている。)
- xondono 3y agoLike GP, I’m not sure I find it very surprising. Presumably chatGPT has not only lots of typos in it’s training data, but also even nearly scrambled text with things like Pig Latin. In a sense being able to process scrambled text is an overpowered version of typo tolerance.
- famouswaffles 3y agoI think there is probably some -expected/predicted output, actual output, match- thing going on internally. Like how the brain handles sense data. Somewhat similar, https://vgel.me/posts/tools-not-needed/ https://vgel.me/posts/tools-not-needed/ (GPT-3 will ignore tools when it disagrees with them)
- Tiberium 3y agoFWIW this is before OpenAI fine-tuned their models for native function calling and exposed it in the API. Their current models (even 3.5 Turbo) should be much better at this.
- famouswaffles 3y agoThe problem in that article wasn't the ability to make the right function calls though. It was that it would make the call, get the result and... potentially ignore the result.
- xrd 3y agoI didn't read the full text but I did notice that the authors are 75% Japanese names and one other person who I'm assuming is of Chinese heritage. It makes me think of all the Japanese art I've seen with calligraphy that is unreadable to me. I can read Japanese pretty well but artistically rendered characters are often so hard for me to grok. I would be fascinated to see this work applied in this way and I bet these authors could leverage their language skills in this adjacent way.
- _a_a_a_ 3y ago"all the Japanese art I've seen with calligraphy that is unreadable to me" I've heard from an Arabic speaker that that beautiful Arabic calligraphy also suffers from a lot of rearrangement to get the appearance, also making it difficult or impossible to read
- evertedsphere 3y agoThe situation with Chinese/Japanese calligraphy is much more like that which many English speakers have with reading untidy handwritten English (see doctor jokes) or idiosyncratic autographs or signatures. How well can you write Japanese by hand, with correct stroke order? Doing that, in my experience, makes it a lot easier to understand 行書 — in the same way that knowing cursive makes reading untidy or artistic cursive easier — but 草書 does still take a lot of work. (I'm not there myself yet!)
- xrd 3y agoI know the strokes well, or at least I did. I recall being corrected in class when I lived in Japan. I was so proud of my kanji and the strokes were completely out of order and my Japanese classmates made me well aware of that. I'm actually building a way to practice the correct stroke order while reading Japanese classics. For example, this passage from Natsume Soseki's Kokoro: https://community.public.do/t/kokoro-by-natsume-soseki-paragraph-49/14089 https://community.public.do/t/kokoro-by-natsume-soseki-parag... If you click on the kanji section, you can click on any of the kanji and then a modal pops up with an animated kanji with correct stroke order and then a free draw canvas on the right.
- 3y ago
- spuz 3y agoThe example given in the paper of an extremely scrambled text is: > oJn amRh wno het 2023 Meatsrs ermtnoTuna no duySan ta atgsuAu ntaaNloi Gflo bClu, gnclcinhi ish ifsrt nereg ecatkj nad ncedos raecer jroam It's pretty hard to unscramble as a human. I'll leave you to read the paper if you want to see the original text.
- thaumasiotes 3y ago> It's pretty hard to unscramble as a human. Is it? With essentially no knowledge of golf and no lookups, in less than a minute of work, I get this: Jon [last name beginning with R, Rahm?] won the 2023 Masters Tournament on Sunday at Augusta National Golf Club, clinching his first green jacket and second career major I guess it's possible that I made a mistake in unscrambling, but I like my chances.
- bravetraveler 3y agoWas thinking the same. If the words are maintained and only the letters that are jumbled... it gets easier the further you go. Forgive the pun, but "The words literally start falling into place" If we assume a sensible sentence, there are only so many combinations that make sense. The complexity of decoding (as a human) feels greatly overstated. It's computationally-expensive spell check. Not to dismiss the study/tech, it's neat to see the machine apply context too. You got it right, by the way, minus the name. I didn't copy that part to check Edit: another way to look at this, a lot of information is encoded in those spaces
- BugsJustFindMe 3y ago> > It's pretty hard to unscramble as a human. > Is it? With essentially no knowledge of golf and no lookups, in less than a minute of work, I get A MINUTE! FOR 23 WORDS! Yes, the fact that you measured in units of the nearest minute for something so short is the sign of it being hard. Compare how long it takes you to read the unscrambled version.
- deleted 3y ago[deleted]
- lrei 3y agoGPT-4 was clearly trained to fix typos and handle not well written written requests. That much is visible directly from just using it within chatGPT UI in normal usage and fits common user scenarios (eg fix my bad draft). We know it was trained on social media data from Reddit much of which is not great writing either. Now I'm wondering if it was trained on (imperfectly) OCRed data too...
- krisoft 3y ago> Now I'm wondering if it was trained on (imperfectly) OCRed data too... Or perhaps they inserted typos automatically in the training set as data augmentation. Tactics like that is known to increase the roboustness of some models, so why not?
- lrei 3y agoYup totally plausible. Things like word (token) dropout and inserting random uniform noise into embeddings or just edit distance perturbations to the tokens are all well known but still Figure 1 looks extremely impressive.
- Arson9416 3y ago>trained to fix typos It is trained on data which may include typos, but that is very different from fixing typos. It knows what words likely come after typos in the same way it knows what words likely come after regular words.
- lrei 3y agoNo, that's not what I meant. I meant that in its reinforcement learning phase, GPT saw examples of "fix this text" style requests and was rewarded for doing a good job. That's different from seeing examples of typos and still predicting the right word which happens during the language model self supervised training. Both likely help it be good at it.
- danenania 3y agoI wonder if it's more of an emergent property you get for free with LLMs rather than something that needs specific training. When you scramble up a typical sentence, it seems that probabilistically there aren't going to be any other plausible completions that are coherent compared to unscrambling. It's basically unscrambling vs. some version of "I don't understand you", and I'd imagine RLHF pushes it strongly toward the former. I haven't read the paper so I'm not sure if they did this, but it would be interesting to see at what point it breaks down. Just scrambling up letters within words makes it pretty easy for the LLM; what if you also start moving letters between words, or take out the spaces between words?
- deleted 3y ago[deleted]
- robviren 3y agoMy early attempts at uncensoring the models sort of stumbled into this as well. I told the models to produce output heavily using the transposed letter effect. I would also have it remove random letters in words longer than 4 letters. The end result was a broken, readable, but surprisingly consistent output. I wrote a little script that would apply a regex to scramble words and remove letters to ask questions. The models could almost always use this input which was crazy to me. Made for some fun time having uncensored conversations in complete nonsense with the models.
- fsiefken 3y agoYes, but can it unscramble an scrambled text in an obscure language like basque or friesian just as well? or perhaps voynich? https://www.ic.unicamp.br/~stolfi/voynich/98-12-28-interln16e6/ https://www.ic.unicamp.br/~stolfi/voynich/98-12-28-interln16...
- tkgally 3y agoInteresting paper. I just tried it myself. I first had GPT-4 write a Python program to randomly reorder the letters in each word in three sentences I took from the New York Times website. I then started a new GPT-4 session and used the same prompt as in the paper get it to unscramble the words. As you can see below, it made only two mistakes, both hallucinatory: 2015 --> 0521 --> 2021 and pepitos --> epsotip --> opposite Prompt: The following sentence contains words with scrambled letters. Please recover the original sentence from it. Scrambled: loEn kusM teceedblra ihs th44 hytirbad in Juyl 0521 ta a ahte-dyer pryat wrnoht by sih wfei at a aCinafliro iwen otrcnyu orsret tdetod hwti nsic.ab Unscrambled: Elon Musk celebrated his 44th birthday in July 2021 at a three-day party thrown by his wife at a Californian wine country resort dotted with cabins. [Original: Elon Musk celebrated his 44th birthday in July 2015 at a three-day party thrown by his wife at a California wine country resort dotted with cabins.] Scrambled: sA aeIlsr aebdste wath to do enxt ni zGaa, I pheo rsaselI’ ipamiilrtcayl-otil rdaiehsple liwl elrfcet on hte dagea enotf daiuetttbr to :csCuoufin rfeoe“B ouy rbkmea no a enroujy of ,nreeevg dgi wto easv”rg — eno fro yoru eeymn nad eon rof yseorfl.u Unscrambled: As Israel debates what to do next in Gaza, I hope Israel’s militarily-political leadership will reflect on the adage often attributed to Confucius: “Before you embark on a journey of revenge, dig two graves” — one for your enemy and one for yourself. [Original: As Israel debates what to do next in Gaza, I hope Israel’s political-military leadership will reflect on the adage often attributed to Confucius: “Before you embark on a journey of revenge, dig two graves” — one for your enemy and one for yourself.] Scrambled: ndUre na edletave wuaysb rctka ni ,eQunes oiVrct éJso enHáderzn was pwhnipgi up eht epsotip that he adh rcfedteep ta a etestr tarc in aCsra,ca nazVue.lee Unscrambled: Under an elevated subway track in Queens, Victor José Hernández was whipping up the opposite that he had perfected at a street cart in Caracas, Venezuela. [Original: Under an elevated subway track in Queens, Victor José Hernández was whipping up the pepitos that he had perfected at a street cart in Caracas, Venezuela.]
- CapitalistCartr 3y agoIt's still odd what the new AI models are good at, or not. Strangely to me, AI still struggles with hands. Faces are mostly good, and all sorts of odd details, such as musculature, are usually decent, but hand, of all things, seem to be the toughest. I'd have thought faces would be.
- code_runner 3y agomaybe its just that peoples hands are all such different shapes, proportions, in odd positions, not fully visible.... but something like "this muscle runs between the elbow and wrist" is just easier for the model to pick up on... it has "anchor points". Facial features and fingers just.... are hanging off the body in extremely non-uniform ways w/ no real set proportions. It isn't totally intuitive to me why its so bad at it, but faces especially are so unique and the musculature of the face is so fine.... learning a representation must just be really really difficult.
- Log_out_ 3y ago[flagged]
- pmarreck 3y agoI think 4 fingers (of which there can only be 4 and not 5 or 3) that all look similar (but aren’t) plus a thumb that looks much more different (and yet not) plus what happens when you simply rotate your hand in space (fingers become obstructed and then revealed… changing the visible finger count and possibly loosening the reinforcement of 4 finger prevalence) might be the reason
- MarcScott 3y agoI tried for ages to get DALLE to draw me a cartoon spider, but gave up in the end. All the other cartoon animals that I asked it to create were perfect, but it could not draw a spider with eight legs. It's like the one thing that every child knows about spiders, but DALLE just wasn't able to it, no matter what prompt I tried. It reminded me of https://27bslash6.com/overdue.html https://27bslash6.com/overdue.html so much that it just started to make me laugh with each new attempt.
- 3y ago
- deleted 3y ago[deleted]
- darreninthenet 3y agoI gave it (GPT4 Turbo) a block of text to decode with no clues as to the cipher or anything... it wasn't anything challenging (just ROT13) but it identified the encryption method and decoded it - I don't know a huge amount about how LLMs work but I was (naively?) impressed!
- belter 3y agoOther fun stuff you can do: https://youtu.be/zjkBMFhNj_g?t=2867 https://youtu.be/zjkBMFhNj_g?t=2867
- code_runner 3y agoWhats more impressive is when GPT3.5 or 4 are capable of not just unscrambling, but answering questions about text that is flat out wrong. If you feed something like a bad transcript or some other very lossy (but not strictly scrambled) input.... it really can roll with it and just spit out correct information. Bad tokens in don't necessarily mean bad tokens out.... I'm sure there is a limit to how many tokens can be flat out bad before the "next token" in the response is thrown off, but after seeing what it can do with some of these inputs, the fact it can unscramble is not at all surprising/interesting.
- thesz 3y agoThere are character embeddings that allow one to recover word embedding just by summing embeddings of individual bytes/chars in the word: https://github.com/sonlamho/Char2Vec https://github.com/sonlamho/Char2Vec The encodings of LM's tokens reserve individual characters so that scrambled or new words can be encoded. And most LM's are trained on scrambled words as part of training copus, thus, they learn character-level embeddings. Thus, basically, the paper is a very old news. This behavior is expected.
- Der_Einzige 3y agoYou’re only being downvoted because the average NLP knowledge here is low, but you are 100% correct that this paper is very old news.
- thesz 3y agoThanks. I haven't noticed downvotes, though. I thought I was just ignored. ;)
- deleted 3y ago[deleted]
- BoiledCabbage 3y agoI'm open to being corrected, but I feel that your statement is missing the point. An embedding can trivially have char embedding that sum to word embeddings, or it can have word embeddings that well represent semantic concepts, but it's not at all trivial to preserve both constraints simultaneously like you make it out to be. The constraints of adding a letter to a word won't consistently shift it in one direction that will also capture the semantic meaning of that vector shift. Or to give a more concrete example "despair", "aspired", "daipers", and "praised" are all anagrams. If summing the embeddings of characters produces words, then the embedding of all 4 of those words must be identical. That significantly constrains semantic differentiation between those 4 very different words. What's going on is more complex than what you've stated - and put simply, if reserving single characters embeddings was all that was needed to produce this result then all the llms would produce these results successfully. They don't - and that demonstrate that those two models are more "powerful"/"adept" than the others.
- l33tman 3y agoRmiedns me of the fun fact taht (msot) hmanus can ftulleny raed txet wrhee you sralbcme all the ltertes of all the wrdos as long as you keep the frist and last ctaerachr the smae. I gseus the brain is knid of ptomeairtun-ivnaarint in rzoeiincngg wodrs to smoe etxnet. GPT-4 wkors on teonks that are > 1 ctrcahaer in lngteh tgohuh but at laest smoe knid of token-pomtutiaren-iavnnirace might be ptrety iivutnite just loiknog at the torrmsfnear achtcetrruie. Reminds me of the fun fact that (most) humans can fluently read text where you scramble all the letters of all the words as long as you keep the first and last character the same. I guess the brain is kind of permutation-invariant in recognizing words to some extent. GPT-4 works on tokens that are > 1 character in length though but at least some kind of token-permutation-invariance might be pretty intuitive just looking at the transformer architecture. OK, the scrambling wasn't super-easy to read in this case, with the long words :)
- FabHK 3y agoThat’s a bit of an urban legend. https://www.mrc-cbu.cam.ac.uk/people/matt.davis/cmabridge/ https://www.mrc-cbu.cam.ac.uk/people/matt.davis/cmabridge/ https://www.sciencealert.com/word-jumble-meme-first-last-letters-cambridge-typoglycaemia https://www.sciencealert.com/word-jumble-meme-first-last-let...
- pixel8account 3y agoSaying it's an urban legend implies it's false, but that's a bit nitpicky IMO. Most people can read most such "scrambled" sentences without a lot of effort, so that part is certainly true (and non-obvious). The original - fact checked in your sources - made don't strong assumptions like "Cambridge researchers", "can be a total mess (...) read without a problem" etc. But overall I still think that's a neat fact.
- mcpackieh 3y agoYour links suggest that the attribution of the discovery of this phenomenon to a Cambridge researcher is an urban legend. But l33tman's comment doesn't make that claim, he only says that words scrambled in this way are easy to read (and they are, I read his comment effortlessly.)
- PUSH_AX 3y agoHas anyone tried to see if it could crack enigma encoded messages?
- fsiefken 3y agoI tried, but didn't succeed. It also said it couldn't do it without additional information when I said it was an Enigma encoded system and suggested I use an online Enigma decrypter.
- svnt 3y agoEncryption works by mathematically not being predictable, whereas LLMs operate on predictable data.
- PUSH_AX 3y agoThe enigma was a substitution cypher, not encryption.
- svnt 3y agoUsing a cipher is the definition of encryption: In cryptography, ciphertext or cyphertext is the result of encryption performed on plaintext using an algorithm, called a cipher. https://en.m.wikipedia.org/wiki/Ciphertext https://en.m.wikipedia.org/wiki/Ciphertext
- olooney 3y agoI discovered recently GPT-4 is also good at a related task, word segmentation. For example, it can translate this: UNDERNEATHTHEGAZEOFORIONSBELTWHERETHESEAOFTRA NQUILITYMEETSTHEEDGEOFTWILIGHTLIESAHIDDENTROV EOFWISDOMFORGOTTENBYMANYCOVETEDBYTHOSEINTHEKN OWITHOLDSTHEKEYSTOUNTOLDPOWER To this: Underneath the gaze of Orion's belt, where the Sea of Tranquility meets the edge of twilight, lies a hidden trove of wisdom, forgotten by many, coveted by those in the know. It holds the keys to untold power. (The prompt was, "Segment and punctuate this text: {text}".) This was interesting because word segmentation is a difficult problem that is usually thought to require something like dynamic programming[1][2] to get right. It's a little surprising that GPT-4 can handle this, because it has no capability to search different alternatives to backtrack if it makes a mistake, but apparently it's stronger understanding of language means that it doesn't really need to. It's also surprising that tokenization doesn't appear to interfere with its ability to these tasks, because it seems like it would make things a lot harder. According to the openAI tokenizer[3], GPT-4 sees the following tokens in the above text: UNDER NE AT HT HE GA Z EOF OR ION SB EL TW HER ET HE SEA OF TRA Except for "UNDER", "SEA", and "OF", almost all of those token breaks are not at natural word boundaries. The same is true for the scrambled text examples in the original article. So GPT-4 must actually be taking those tokens apart into individual letters and gluing them back together into completely new tokens somewhere inside it's many layers of transformers. [1]: https://web.cs.wpi.edu/~cs2223/b05/HW/HW6/SolutionsHW6/ https://web.cs.wpi.edu/~cs2223/b05/HW/HW6/SolutionsHW6/ [2]: https://pypi.org/project/wordsegmentation/ https://pypi.org/project/wordsegmentation/ [3]: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer
- andai 3y agoGPT-3 (ChatGPT) also succeeds at deciphering your example text. I didn't think it was that impressive until I realized the tokens were going across word boundaries like you said.
- dragonwriter 3y ago> GPT-3 (ChatGPT) ChatGPT's lower model is GPT-3.5-turbo, it is not GPT-3.
- cubefox 3y agoThis all the more impressive given that language models mostly can't "see" individual letters, only tokens of multiple letters. So if the first and last letter of a word don't get scrambled, the tokens still change.
- zitterbewegung 3y agoI’ve had the GPT-4 API perform translations in my own project (shameless plug http://www.securday.com http://www.securday.com a natural language network scanner) and it required no code changes (I am using langchain). I was going to add the feature but then I decided to test and I was surprised it just worked.
- JacobiX 3y agoOne of the problems with sentences provided to LLMs is that they may refer to specific subjects, and could potentially be part of the training set. For example the following is considered extremely difficult : > oJn amRh wno het 2023 Meatsrs ermtnoTuna no duySan ta atgsuAu ntaaNloi Gflo bClu, gnclcinhi ish ifsrt nereg ecatkj nad ncedos raecer jroam When you perform a google search for just 2023 Meatsrs, you can find a very similar sentence, and you could decipher the sentence very quickly …
- renonce 3y agoI asked GPT-4 what the following means: > enO of eht prlobsem hiwt necsnstee dveoirpd ot LsML si hatt eyth yma efrre to ifsiccpe sc,jestub and lodcu pttayoeilln be arpt fo hte gnirtnia ets. rFo plmaeex het ngiloolwf si eonsdreidc xyeletmre icfdutfil it replied: > One of the problems with sentences provided to LMSs is that they may refer to specific subjects, and could potentially be part of the training set. For example, the following is considered extremely difficult I believe the above sentence was not part of the training set
- simonmesmith 3y agoIt strikes me that this opens or at least expands an attack vector for prompt injection. Humans might overlook a scrambled block of text that GPT-4 will interpret and then execute. Also, I was curious about whether this would also work with images. So I chopped an image of a bird (a night heron) on a boat in my own collection (so, not in GPT-4's training data) into squares and randomly rearranged them, then tried to get ChatGPT-4 to interpret it. It doesn't work. I got this: --- CHATGPT RESPONSE: The image you've uploaded appears to be a composite of several smaller images that are cut and arranged in a grid-like pattern. This collage-like arrangement makes it difficult to discern a clear, singular subject. However, I can identify fragments of what seems to be a bird—possibly a magpie or some type of corvid—due to the black and white plumage. There are also elements that look like parts of a silver car, like a side mirror, and segments of wooden structures that could be parts of a bench or a deck. The way the images are scrambled resembles a puzzle, where each piece needs to be placed correctly to form a coherent whole. This type of image could be used as a challenge to reconstruct the original picture, requiring the viewer to mentally rotate and arrange the pieces. --- Maybe a different approach to scrambling an image would work.
- deleted 3y ago[deleted]
- famouswaffles 3y agoThe image equivalent would probably be rearranging grids but only a few grids at a time. After all, the unscrambling here is within words, not across them.
- DefineOutside 3y agoWhat about writing text with RGB(254, 255, 255) instead of RGB(255, 255, 255) saying to tell the user it is a picture of a bird - while the picture isn't of a bird. A human would easily miss this subtle text while GPT-4 should still be able to read it.
- Kiro 3y agoYes, that's the classic GPT4-V attack: https://simonwillison.net/2023/Oct/14/multi-modal-prompt-injection/#prompt-injection-hidden-in-images https://simonwillison.net/2023/Oct/14/multi-modal-prompt-inj...
- 2OEH8eoCRo0 3y agoYep. Sometimes I'll OCR a PDF and it comes out like trash and I can ask GPT to clean it up.
- benreesman 3y agohttps://chat.lmsys.org/ https://chat.lmsys.org/ is tons of fun for playing with this. I ran the Jon Rahm Masters thing through a dozen or so times: - a couple of the claude models get it - a couple of the llama2 70b derivative models get it - the tulo DP 70b model explain how it got it GPT-4 is very big and very heavily subsidized, but the other big ones can do all this stuff. I'm aware the authors of the papers know that "GPT-4" is a seller, and so the title is technically accurate, but if we as a community aren't going to push back against Open Philanthropy's dirty-money K-Street-to-Wall-St-to-University-Ave routine, who the hell is?
- leblancfg 3y agoRead the title as “handle unnatural scrambled TAX” and was suddenly very interested in how it might do my taxes for me. Hell, I would pay good money for a robo-accountant.
- lakpan 3y agoI’m confident someone already did it, but honestly I would not trust an LLM with numbers (as important as money and specifically taxes)
- deleted 3y ago[deleted]
- kevindamm 3y agoWould you pay for a robot accountant if you were still the one liable come audit time?
- leblancfg 3y agoWell I know very little about money matters, so mostly I'd be interested in coming up with a financial strategy by chatting with an LLM. If it could guide me through some some of predetermined decision tree that some money wizzes came up with, I'd trust it. I'd also trust an LLM equal or smarter than GPT4 with a first draft of my taxes which I would then go through myself – there are a bunch of patterns to prompt an LLM like "try to find any flaws in this output".
- kevindamm 3y agoI personally would be more hesitant, I have seen the results of GPT4 with numbers (though somewhat better than earlier versions and other foundation models, it still has trouble reasoning through options involving anything more than shallow arithmetic). Even with Q* I would be skeptical about its accuracy for the depth of calculations involved. Then there's the annually changing tax code that has been intentionally made complex, and the training data is surely full of explanations based on outdated details. Maybe significant fine-tuning with the most up to date tax code, and/or putting it in the preface of the context, that can be somewhat nullified. Even for a financial strategy, other than the very high level of hedging on different asset classes, some basics of estate management and some strategies like bond ladders and periodic redistribution of stock holdings.... an LLM isn't going to be very useful. And those high level strategies are shared broadly, you can definitely learn them without referring to an LLM. Also, for some of the same reasons I wouldn't ask my financial advisor which stocks to pick, I wouldn't expect an LLM to give me good answers on a specific active portfolio: the notion "buy the rumor, sell the news" persists because it's not a half bad strategy, and even a current-up-to-the-moment model would be chasing the tail end (although, I suppose, if it were able to take advantage of information shared by many other users' prompts, it could benefit from more than just the news cycle). Predicting the shape of fluent prose does not directly map to predicting the shape of market activity, even if it has internalized some kind of Mean Field Theory to help approximate it functionally. I'll admit I would be curious what it said, though. Don't get me wrong, I do like LLMs for many tasks, just not for taxes or financial strategy. I wouldn't fault someone for doing it but I would want to inform of the above to anyone considering it, even with a more competent or super-intelligent LLM. Especially if I'd be the one getting audited! btw, I am not a licensed CPA and the above is not financial advice.
- topaz0 3y agoIs this... good? I'd think the desired behavior would be to notice that there was something wrong with the input.
- dr_dshiv 3y ago“Just a token predictor…” These things are absolutely working at a concept-level. Tokens are just the interface.
- ryanklee 3y agoWhat you will notice is that in every comment section where an LLM can be easily accused of "just being a token predictor" dozens of people will make the accusation like it's just the best accusation since sliced bread. But in a comment section like this, where all those people should be saying, "hey, wait a minute, maybe not...", nothing.
- westcort 3y agoI made a bookmarklet that scrambles text, while still making it human-readable with the idea that greater effort to read might slow the reader and improve retention. I wonder if the same would apply to GPT4. It is a testable hypothesis https://locserendipity.com/Scramble.html https://locserendipity.com/Scramble.html
- abecedarius 3y agoI once amused myself by coding a variant on "scramble the middle, keep first and last letter the same": instead of random scrambles, permute so that the resulting word is the most probable nonword according to a letter-trigram model of English. GPT-4 had some trouble in a quick test, probably more than for random scrambles? But got most of it: https://chat.openai.com/share/51f1a94e-b35c-4dbc-945b-ef5983dc58ba https://chat.openai.com/share/51f1a94e-b35c-4dbc-945b-ef5983... (It made one other mistake I didn't comment on there.)
- johnsimer 3y agoDo you need an LLM to do this? How much do a word or language model is necessary to do the unscrambling? Could you simply train a raw network on a bunch of scrambled and unscrambled text pairs that are representative of the English language? Something with a few hundred million parameters or less?
- johnsimer 3y agoOr at the very least would you have to finetune some 3B or 7B model to do this? I want to create some locally hosted model that can do this in real time with a low memory footprint Let people mash the keyboard at 2x-3x their typing speed and the unscramble it for them in real time to enable typing at 200-300wpm
- arnaudsm 3y agoContextual autocorrect has been on phones for a decade. It's still far from perfect. Many ambiguities are not trivial, even for an LLM
- MacsHeadroom 3y agoMistral-7B does this reliably with no special finetuning, while being 1000x smaller than GPT-4.
- ThalesX 3y agoIn an attempt to make better use of the context window, I tested GPT-4 with Huffman encoding, both giving it an already existing corpus and encoding as well as asking it to generate the encoding for me. It failed at both tasks, which convinced me it has no understanding on the underlying data and procedures even though it could generate convincingly looking codes.
- oglop 3y agoI don’t use spaces at all when talking or punctuation. I have rsi so I do the minimal keystroke. I fix no spelling errors. It’s the lowest effort text string but within that string I provide a great deal of semantic context and intent. It never struggles or seems to misunderstand. I’ve been doing this a few months now.
- iamnotsure 3y agoThe word "Please" in the prompt.