5 ms·

Another thing that LLMs are (surprisingly?) good at is randomly mixed language conversations, here's something I tried a few days ago: Prompt: Kan du comprende

Another thing that LLMs are (surprisingly?) good at is randomly mixed language conversations, here's something I tried a few days ago: Prompt: Kan du comprender questo text? Anata wa quelque inkling habe about lo que se parla dans la presente oraçao? Repondez med sam way of goboriu. ChatGPT: Ja, ich kann verstehen este mix de idiomas! Tu es usando une combinación de várias línguas pour expresar una pregunta. Je vais responder en the same modo: Si tu veux kommunicieren en cette manière, c'est possible, aber il peut être ein bisschen confusing pour andere people! 你觉得这个方式有趣吗？ O préfères-tu un idioma más claro? Deepseek: Ja, jeg kan forstå en lille bit av ciò que tu está dizendo. Anata no message wa molto interessante, pero un poco confusing aussi. Si tu peux écrire in una lingua sola, sería plus facile para mi at comprender. Danke, merci, arigatou! Claude: Oui, je kan entender algo of this texte mezclado! C'est un interessante modo de communicare using différentes languages together. Je trouve ça quite divertido y puedo responder en la misma manera mixta. Ist das what you were looking for? Watashi wa happy to continue questa conversation in whatever linguistic style anata wa prefer! Perplexity: Ja, ich kann diesen Text verstehen! Vous avez créé una frase muy interesante con palabras from different languages. Let me rispondere using a similar mishmash of lingue: Questo è un esempio fascinante di code-switching e language mixing. Watashi wa impressed by la creatività y el modo en que vous avez combiné various idiomas in one sentence. Es ist ein poco challenging zu decifrar, aber anche molto divertente! Ich würde sagen que el mensaje principal è about compréhension linguistique et la capacité de entender verschiedene Sprachen when they're mixed together. C'est une manière intéressante de tester language skills et cognitive flexibility. Grazie for questa stimolante question! J'espère que mi respuesta was equally entertaining und in lo mismo estilo multilingüe.

deleted 2y ago
[deleted]
mistrial9 2y ago
this fits the supposition -- since LLMs can be fed patterns of nonsense and learn to reply in those patterns, LLMs are not intelligent. YNews corollary : since posters cannot resist making new swathes of Look At This LLM Output, the open nature of tech boards is doomed in some ways (?)
- fcatalan 2y ago
  Those texts aren't nonsense. The prompt has a meaning, the LLMs are able to understand it, and are able to reply with coherent and understandable responses crafted in the same way the prompt was written. For me it's a very clear example of something that is very far from any training data coming out of the models. Intelligent? No, but for me it points to the idea that "language is solved".
  - TechDebtDevin 2y ago
    language will be solved when LLMs are translating Whale's songs to human language imo.
  - TechDebtDevin 2y ago
    As a Vegan, maybe I'm a little biased, but I often think about what the implications of a universal translator would be, if it did infact give us the ability to understand animals. What would that imply if you could drive by a slaughterhouse and be able to understand animals saying goodbye to their loved ones... assuming this is happening.. Would all slaughtering stop? Or would people be okay with that? Interesting times ahead if there is any possibility for ML to translate animal language.
    - stavros 2y ago
      We don't slaughter animals because we think they don't mind dying, we slaughter them because we've outsourced the mass killings to people who don't mind doing it, and a steak looks enough unlike a cow that we don't think that it used to be alive. Basically, if we had to slaughter our own cows, I doubt we'd be eating as much meat.
      - bongodongobob 2y ago
        I can tell you've never lived in the Midwest, or maybe just not outside of a city. People have dedicated chest freezers for wild game that they keep full all year. Opening of hunting and fishing seasons are huge deals.
pbhjpbhj 2y ago
Word vectors for the same concept in different languages are very close, that's not really surprising. The whole `v_king - v_male + v_female = v_queen` idea leads to an intuition that `v_grazie - v_italien + v_deutsch = v_danke`. Code switching probably appears a lot in both literature and audio from some contexts too. Why would it matter that a concept is expressed in a different human language? How would you structure a multilingual LLM that it couldn't syncretise languages in this way? You'd seemingly need to properly segregate the languages.
- fcatalan 2y ago
  Yes I understand the encodings will be close and that helps, I guess that's why they produce good translations, but I'm intrigued by the LLM having so much control of the switching without even explicit prompting, just a one-shot example. I also guess I'm easily impressed.
- nine_k 2y ago
  I suppose this mechanism, only quite a bit up the conceptual ladder, is necessary for LLMs to be able to translate between languages, which they apparently are trained to do, explicitly or not.
foolswisdom 2y ago
I think the research by anthropic released recently showed that language is handled independently of the "concepts" they convey, so first you get the concepts, then you get the translation to language.
ashirviskas 2y ago
I've only taken duolingo in French for a few months a few years ago, have heard my girlfriend practice her Italian and I've spent some time around german people. Had Russian lessons and I have pretty good English and Lithuanian skills. I'm only fluent in the last two languages. I still understood most of your prompt. So I don't think this is a good test. Reading that prompt again, I think watching some anime with subs helped too.
- vidarh 2y ago
  Yeah, I read English, German, French, and the Scandinavian languages, and beyond that Italian and Spanish only via similarity to French and the presence of Latin in the others listed, and that was enough to read it at near full speed.
torben-friis 2y ago
Yup, LLMs are a polyglot’s dream interface, considering multi language is a feature that pretty much all companies screw up each in their own way. And then there’s apple, which will not let me use their AI features because Siri needs to be in the same language as iOS, Siri is set to English and iOS is set to “English (Spain)” (????).
- hunter2_ 2y ago
  I tried putting a few of GP's multilingual paragraphs into google translate on detect mode, and it got everything into English perfectly! Interestingly, it declares a single language as having been detected, which varies perhaps based on majority input language.
kranner 2y ago
Mixed scripts as well. In March 2024 I asked Gemini Advanced (whatever the version was at the time) to transliterate an image which had the following Persian text on it: > یوسفی بود ولی هیچ خریدار نداشت Its output was: > Yoosefi بود ولی هیچ khaरीदār nadāsht That's three different scripts with two different Romanisation schemes just for the Latin/Roman script (writing "Yoosefi" as "Yūsefī" or "Yūsufī" would have been more consistent with "nadāsht").
nine_k 2y ago
Oh, this is a mental virus stronger than Ghiblifying all the things. Alas, ahora mina wa is perdú. Él kite iru.
shmeeed 2y ago
I get strong Belter Creole vibes from this one

deleted 2y ago

[deleted]

mistrial9 2y ago

this fits the supposition -- since LLMs can be fed patterns of nonsense and learn to reply in those patterns, LLMs are not intelligent. YNews corollary : since posters cannot resist making new swathes of Look At This LLM Output, the open nature of tech boards is doomed in some ways (?)

fcatalan 2y ago
Those texts aren't nonsense. The prompt has a meaning, the LLMs are able to understand it, and are able to reply with coherent and understandable responses crafted in the same way the prompt was written. For me it's a very clear example of something that is very far from any training data coming out of the models. Intelligent? No, but for me it points to the idea that "language is solved".
- TechDebtDevin 2y ago
  language will be solved when LLMs are translating Whale's songs to human language imo.
- TechDebtDevin 2y ago
  As a Vegan, maybe I'm a little biased, but I often think about what the implications of a universal translator would be, if it did infact give us the ability to understand animals. What would that imply if you could drive by a slaughterhouse and be able to understand animals saying goodbye to their loved ones... assuming this is happening.. Would all slaughtering stop? Or would people be okay with that? Interesting times ahead if there is any possibility for ML to translate animal language.
  - stavros 2y ago
    We don't slaughter animals because we think they don't mind dying, we slaughter them because we've outsourced the mass killings to people who don't mind doing it, and a steak looks enough unlike a cow that we don't think that it used to be alive. Basically, if we had to slaughter our own cows, I doubt we'd be eating as much meat.
    - bongodongobob 2y ago
      I can tell you've never lived in the Midwest, or maybe just not outside of a city. People have dedicated chest freezers for wild game that they keep full all year. Opening of hunting and fishing seasons are huge deals.

pbhjpbhj 2y ago

Word vectors for the same concept in different languages are very close, that's not really surprising. The whole `v_king - v_male + v_female = v_queen` idea leads to an intuition that `v_grazie - v_italien + v_deutsch = v_danke`. Code switching probably appears a lot in both literature and audio from some contexts too. Why would it matter that a concept is expressed in a different human language? How would you structure a multilingual LLM that it couldn't syncretise languages in this way? You'd seemingly need to properly segregate the languages.

fcatalan 2y ago
Yes I understand the encodings will be close and that helps, I guess that's why they produce good translations, but I'm intrigued by the LLM having so much control of the switching without even explicit prompting, just a one-shot example. I also guess I'm easily impressed.
nine_k 2y ago
I suppose this mechanism, only quite a bit up the conceptual ladder, is necessary for LLMs to be able to translate between languages, which they apparently are trained to do, explicitly or not.

foolswisdom 2y ago

I think the research by anthropic released recently showed that language is handled independently of the "concepts" they convey, so first you get the concepts, then you get the translation to language.

ashirviskas 2y ago

I've only taken duolingo in French for a few months a few years ago, have heard my girlfriend practice her Italian and I've spent some time around german people. Had Russian lessons and I have pretty good English and Lithuanian skills. I'm only fluent in the last two languages. I still understood most of your prompt. So I don't think this is a good test. Reading that prompt again, I think watching some anime with subs helped too.

vidarh 2y ago
Yeah, I read English, German, French, and the Scandinavian languages, and beyond that Italian and Spanish only via similarity to French and the presence of Latin in the others listed, and that was enough to read it at near full speed.

torben-friis 2y ago

Yup, LLMs are a polyglot’s dream interface, considering multi language is a feature that pretty much all companies screw up each in their own way. And then there’s apple, which will not let me use their AI features because Siri needs to be in the same language as iOS, Siri is set to English and iOS is set to “English (Spain)” (????).

hunter2_ 2y ago
I tried putting a few of GP's multilingual paragraphs into google translate on detect mode, and it got everything into English perfectly! Interestingly, it declares a single language as having been detected, which varies perhaps based on majority input language.

kranner 2y ago

Mixed scripts as well. In March 2024 I asked Gemini Advanced (whatever the version was at the time) to transliterate an image which had the following Persian text on it: > یوسفی بود ولی هیچ خریدار نداشت Its output was: > Yoosefi بود ولی هیچ khaरीदār nadāsht That's three different scripts with two different Romanisation schemes just for the Latin/Roman script (writing "Yoosefi" as "Yūsefī" or "Yūsufī" would have been more consistent with "nadāsht").

nine_k 2y ago

Oh, this is a mental virus stronger than Ghiblifying all the things. Alas, ahora mina wa is perdú. Él kite iru.

shmeeed 2y ago

I get strong Belter Creole vibes from this one