3 ms·

You assume that existing ones can’t? Using the example provided in the main link, ChatGPT-4 had no trouble at all: https://chat.openai.com/share/3b8faf2e-4eca-

You assume that existing ones can’t? Using the example provided in the main link, ChatGPT-4 had no trouble at all: https://chat.openai.com/share/3b8faf2e-4eca-4d52-97fb-ac89a4923955 https://chat.openai.com/share/3b8faf2e-4eca-4d52-97fb-ac89a4...

devit 3y ago
Seems ineffective with abstract concepts. "In mathematics, a real number is a number that can be used to measure a continuous one-dimensional quantity such as a distance, duration or temperature. Here, continuous means that pairs of values can have arbitrarily small differences. Every real number can be almost uniquely represented by an infinite decimal expansion." -> "toki pi nanpa, nanpa lon li nanpa li ken lukin e suli tawa wan. lon ni la, nanpa tu li ken ala li poka li ken lili. nanpa lon ali li ken lukin kepeken nimi nanpa mute tawa pini ala" -> "Talking about numbers, when a number is present, it can be seen to be important or significant to an individual." "In this context, the number two might not be significant or might be seen as small or lesser." "All numbers can be seen using many number names endlessly or without an end." ChatGPT 4, obviously in separate chats: as you can see a complete failure to convey and roundtrip the original message.
- coder543 3y ago
  > Seems ineffective with abstract concepts I’ve only heard of Toki Pona for about 20 minutes, but everything I’m reading says that’s just a normal property of the language, not a limitation of ChatGPT. As a human, which of the 120 Toki Pona words would you confidently and correctly translate as “continuous” or “real” or any of these other linguistically uncommon concepts? Based on my own chat with ChatGPT, I feel almost certain that it warned you about this exact limitation of the language, but you ignored that?
  - devit 3y ago
    "continuous" (in this actually incorrect usage, since the correct term is "dense") -> said of an algebraic structure such that for every x != y there exists z such that x < z and z < y -> characteristic ("nimi"?) of a set ("poki") where for ("tawa?") all ("ale") thing ("ijo") different ("ante" + "lon"?) than other ("ante" + "nimi"?) thing ("ito") there exists ("lon") a new ("sin") thing ("ijo") such that the first ("wan" + "nimi" + "ito"?) thing is less ("lili") than the new thing ("sin" + "ito"?) and the new thing ("sin" + "ito"?) is less ("lili") that the second thing ("tu" + "nimi" + "ito"?) Would need some way to join into a grammatical structure and probably a preamble that establishes that the context is mathematics.
- tbodt 3y ago
  I would definitely not attempt to translate something like that in one shot - I'd read it and digest it for a bit and come up with a way to say the same thing without being unnecessarily obtuse, Thing Explainer style. mi nanpa e mute ijo la kipisi ijo li ken ala lon. ijo wan sin li kama la mi suli wan e nanpa. taso mi ken ala suli kipisi e nanpa. nanpa pi mute ijo la ni li ken ala: ijo tu en kipisi pi ijo wan li lon. taso mi nanpa e suli ijo la kipisi nanpa ale li ken. ijo li kama suli sin pi kipisi wan la mi ken suli pi kipisi wan e nanpa. kipisi li ken lili ale. nasin nanpa ni li ken nanpa e suli ijo e suli tenpo e suli seli. nanpa pi suli ijo la ni li ken: mi sitelen e nanpa la sitelen ni li ken nanpa ni taso li sama ala sitelen pi nanpa ante. taso nanpa mute la sitelen ona li suli ale li ken ala pini. GPT3 doesn't translate this very well, it seems to miss the fine points of grammar, but maybe GPT4 would do better
  - coder543 3y ago
    This is what GPT4 came up with for the translation: > The text you provided is in Toki Pona, a constructed language with a minimalistic vocabulary. The translation to English is challenging due to the highly context-dependent and simplified nature of the language. Without further context, here's a rough translation: > "When I count many things, cutting things is not possible. When one new thing comes, I increase the count by one. But I can't increase the count by cutting it. In the case of counting many things, this is not possible: two things and a cut of one thing exist. But when I count large things, cutting all numbers is possible. When something becomes large by cutting one, I can increase the count by cutting one. Cutting can make everything small. This counting method can count large things, long times, and hot things. In the case of counting large things, it's possible: when I write a number, this writing can count it but is not the same as writing a different number. But with many numbers, its writing is always large and cannot end." > Please note that the translation might not capture all the nuances, and the interpretation may vary based on the context in which the text is used.
    - tbodt 3y ago
      Much closer
clankyclanker 3y ago
This is precisely the case where ChatGPT (or any LLM) will have trouble that a layperson would have trouble validating: Toki Pona has a limited document corpus and requires contextual interpretation obvious to experts. In short, expect hallucinations and other lies.
- coder543 3y ago
  In this case, I validated against the hand-translated example in the main link. It did fine. This is probably easy mode for it, given the tiny vocabulary at hand, even though it has surely seen very little Toki Pona in the wild. Large language models are extremely good at translation. I’ve personally tested ChatGPT-4 against Google Translate and other translators on some hard translations (languages that are not similar to English, such as Chinese), and it has consistently provided better translations than Google Translate in my opinion. As an example, for validation purposes, news stories that have been reported worldwide make things easier, since you can establish the facts of the translation by hand, and then compare what comes out. What does your comment actually add to the discussion? Yes, ChatGPT can be wrong. So can Google Translate! If you’d like to provide a clear example of ChatGPT-4 (not ChatGPT-3.5) making up a Toki Pona translation that is completely disconnected from reality, please do. That would be legitimately interesting conversational material. General cynicism isn’t. ChatGPT absolutely has limitations, and translation is a hard problem, even for the industry-standard tools. Crossvalidation with multiple translation tools and ideally humans is important for any translations that actually matter.
  - NoZebra120vClip 3y ago
    Since LLMs' only valid uses involve entertainment and creativity, and Toki Pona is more or less a recreational language without serious application, I think they go together like two peas in a pod.
    - MostlyStable 3y ago
      >only valid use... Well. That's an opinion I suppose. -edit- After posting, I decided this was too snarky/low value on it's own. So to add: I personally get quite a bit of value out of it as an ad-hoc German tutor and as one-among-many tools for research and learning on new topics for which I am not already an expert. I've also used it to help me make and troubleshoot very simple code in languages that I am not an expert in. I personally have trivially found lots of uses that probably all cluster around "educational" to one degree or another. Is it perfect? Absolutely not. Does it hallucinate/get things wrong? Absolutely. Is it useful despite those limitations? Emphatically yes. This idea that its "only valid use" is entertainment doesn't even make sense to me. "Entertainment" is perhaps where I find it the most deficient relative to other current options.
Aachen 3y ago
This gives a 404 for me
- coder543 3y ago
  Sorry, I had just deleted it from my history without realizing that it would break the share link. Updated the link with a new session demonstrating the same thing now.
Rebelgecko 3y ago
What about going the other way? I'm on chat gpt3.5 FWIW, but when I ask it to translate simple phrases from English to Toki Pona (electric car, barking dog, etc) it hallucinates words that aren't in the ~120 word dictionary I found.
- coder543 3y ago
  As you noticed, ChatGPT3.5 is extremely prone to hallucination. It can still happen on ChatGPT4, but it is much less frequent, in my experience. Here is what I got on the first try: https://chat.openai.com/share/88b0358e-c96d-4880-b939-7c62cf8b6127 https://chat.openai.com/share/88b0358e-c96d-4880-b939-7c62cf... The answers seem plausible, and the words are actual Toki Pona words. My special instructions have nothing to do with language or translation, but for the sake of transparency I did it a second time with custom instructions disabled: https://chat.openai.com/share/b85a59d9-2678-45c8-8c2e-8b8a80577441 https://chat.openai.com/share/b85a59d9-2678-45c8-8c2e-8b8a80... Regardless, there is a very limited dictionary to work with in Toki Pona, so communicating extremely specific ideas seems like it is always going to be problematic. The language doesn’t even have a specific word for “dog”.

devit 3y ago

Seems ineffective with abstract concepts. "In mathematics, a real number is a number that can be used to measure a continuous one-dimensional quantity such as a distance, duration or temperature. Here, continuous means that pairs of values can have arbitrarily small differences. Every real number can be almost uniquely represented by an infinite decimal expansion." -> "toki pi nanpa, nanpa lon li nanpa li ken lukin e suli tawa wan. lon ni la, nanpa tu li ken ala li poka li ken lili. nanpa lon ali li ken lukin kepeken nimi nanpa mute tawa pini ala" -> "Talking about numbers, when a number is present, it can be seen to be important or significant to an individual." "In this context, the number two might not be significant or might be seen as small or lesser." "All numbers can be seen using many number names endlessly or without an end." ChatGPT 4, obviously in separate chats: as you can see a complete failure to convey and roundtrip the original message.

coder543 3y ago
> Seems ineffective with abstract concepts I’ve only heard of Toki Pona for about 20 minutes, but everything I’m reading says that’s just a normal property of the language, not a limitation of ChatGPT. As a human, which of the 120 Toki Pona words would you confidently and correctly translate as “continuous” or “real” or any of these other linguistically uncommon concepts? Based on my own chat with ChatGPT, I feel almost certain that it warned you about this exact limitation of the language, but you ignored that?
- devit 3y ago
  "continuous" (in this actually incorrect usage, since the correct term is "dense") -> said of an algebraic structure such that for every x != y there exists z such that x < z and z < y -> characteristic ("nimi"?) of a set ("poki") where for ("tawa?") all ("ale") thing ("ijo") different ("ante" + "lon"?) than other ("ante" + "nimi"?) thing ("ito") there exists ("lon") a new ("sin") thing ("ijo") such that the first ("wan" + "nimi" + "ito"?) thing is less ("lili") than the new thing ("sin" + "ito"?) and the new thing ("sin" + "ito"?) is less ("lili") that the second thing ("tu" + "nimi" + "ito"?) Would need some way to join into a grammatical structure and probably a preamble that establishes that the context is mathematics.
tbodt 3y ago
I would definitely not attempt to translate something like that in one shot - I'd read it and digest it for a bit and come up with a way to say the same thing without being unnecessarily obtuse, Thing Explainer style. mi nanpa e mute ijo la kipisi ijo li ken ala lon. ijo wan sin li kama la mi suli wan e nanpa. taso mi ken ala suli kipisi e nanpa. nanpa pi mute ijo la ni li ken ala: ijo tu en kipisi pi ijo wan li lon. taso mi nanpa e suli ijo la kipisi nanpa ale li ken. ijo li kama suli sin pi kipisi wan la mi ken suli pi kipisi wan e nanpa. kipisi li ken lili ale. nasin nanpa ni li ken nanpa e suli ijo e suli tenpo e suli seli. nanpa pi suli ijo la ni li ken: mi sitelen e nanpa la sitelen ni li ken nanpa ni taso li sama ala sitelen pi nanpa ante. taso nanpa mute la sitelen ona li suli ale li ken ala pini. GPT3 doesn't translate this very well, it seems to miss the fine points of grammar, but maybe GPT4 would do better
- coder543 3y ago
  This is what GPT4 came up with for the translation: > The text you provided is in Toki Pona, a constructed language with a minimalistic vocabulary. The translation to English is challenging due to the highly context-dependent and simplified nature of the language. Without further context, here's a rough translation: > "When I count many things, cutting things is not possible. When one new thing comes, I increase the count by one. But I can't increase the count by cutting it. In the case of counting many things, this is not possible: two things and a cut of one thing exist. But when I count large things, cutting all numbers is possible. When something becomes large by cutting one, I can increase the count by cutting one. Cutting can make everything small. This counting method can count large things, long times, and hot things. In the case of counting large things, it's possible: when I write a number, this writing can count it but is not the same as writing a different number. But with many numbers, its writing is always large and cannot end." > Please note that the translation might not capture all the nuances, and the interpretation may vary based on the context in which the text is used.
  - tbodt 3y ago
    Much closer

clankyclanker 3y ago

This is precisely the case where ChatGPT (or any LLM) will have trouble that a layperson would have trouble validating: Toki Pona has a limited document corpus and requires contextual interpretation obvious to experts. In short, expect hallucinations and other lies.

coder543 3y ago
In this case, I validated against the hand-translated example in the main link. It did fine. This is probably easy mode for it, given the tiny vocabulary at hand, even though it has surely seen very little Toki Pona in the wild. Large language models are extremely good at translation. I’ve personally tested ChatGPT-4 against Google Translate and other translators on some hard translations (languages that are not similar to English, such as Chinese), and it has consistently provided better translations than Google Translate in my opinion. As an example, for validation purposes, news stories that have been reported worldwide make things easier, since you can establish the facts of the translation by hand, and then compare what comes out. What does your comment actually add to the discussion? Yes, ChatGPT can be wrong. So can Google Translate! If you’d like to provide a clear example of ChatGPT-4 (not ChatGPT-3.5) making up a Toki Pona translation that is completely disconnected from reality, please do. That would be legitimately interesting conversational material. General cynicism isn’t. ChatGPT absolutely has limitations, and translation is a hard problem, even for the industry-standard tools. Crossvalidation with multiple translation tools and ideally humans is important for any translations that actually matter.
- NoZebra120vClip 3y ago
  Since LLMs' only valid uses involve entertainment and creativity, and Toki Pona is more or less a recreational language without serious application, I think they go together like two peas in a pod.
  - MostlyStable 3y ago
    >only valid use... Well. That's an opinion I suppose. -edit- After posting, I decided this was too snarky/low value on it's own. So to add: I personally get quite a bit of value out of it as an ad-hoc German tutor and as one-among-many tools for research and learning on new topics for which I am not already an expert. I've also used it to help me make and troubleshoot very simple code in languages that I am not an expert in. I personally have trivially found lots of uses that probably all cluster around "educational" to one degree or another. Is it perfect? Absolutely not. Does it hallucinate/get things wrong? Absolutely. Is it useful despite those limitations? Emphatically yes. This idea that its "only valid use" is entertainment doesn't even make sense to me. "Entertainment" is perhaps where I find it the most deficient relative to other current options.

Aachen 3y ago

This gives a 404 for me

coder543 3y ago
Sorry, I had just deleted it from my history without realizing that it would break the share link. Updated the link with a new session demonstrating the same thing now.

Rebelgecko 3y ago

What about going the other way? I'm on chat gpt3.5 FWIW, but when I ask it to translate simple phrases from English to Toki Pona (electric car, barking dog, etc) it hallucinates words that aren't in the ~120 word dictionary I found.

coder543 3y ago
As you noticed, ChatGPT3.5 is extremely prone to hallucination. It can still happen on ChatGPT4, but it is much less frequent, in my experience. Here is what I got on the first try: https://chat.openai.com/share/88b0358e-c96d-4880-b939-7c62cf8b6127 https://chat.openai.com/share/88b0358e-c96d-4880-b939-7c62cf... The answers seem plausible, and the words are actual Toki Pona words. My special instructions have nothing to do with language or translation, but for the sake of transparency I did it a second time with custom instructions disabled: https://chat.openai.com/share/b85a59d9-2678-45c8-8c2e-8b8a80577441 https://chat.openai.com/share/b85a59d9-2678-45c8-8c2e-8b8a80... Regardless, there is a very limited dictionary to work with in Toki Pona, so communicating extremely specific ideas seems like it is always going to be problematic. The language doesn’t even have a specific word for “dog”.