3 ms·

The blog post mentions that you have to double check everything, but they did not check the translation. If you just put the same text through multiple times (i

by sp332 3y ago

The blog post mentions that you have to double check everything, but they did not check the translation. If you just put the same text through multiple times (in different chat sessions), you get completely different answers back. I think it's just making up the whole thing.

AlotOfReading 3y ago
It is. Here's a professional translation for this [0]: ...they made bricks. I razed that small palace in its entirety, took a large area from the fields for an addition, and added (it) to it (the palace). I laid its foundations with limestone, strong stone from the mountains, and raised the terrace. I summoned the kings of Ḫatti and Across the River (Syria-Palestine) Baʾalu, king of Tyre, Manasseh, king of Judah, Qaʾuš-gabri, king of Edom, Muṣurī, king of Moab, Ṣil-Bēl, king of Gaza, Mitinti, king of Ashkelon, ... Even if the translation wasn't very wrong, I'd be incredibly skeptical of this because Akkadian and Sumerian share a lot of words that are written the same way, but have very different meanings (which can even be mixed within a document!). They're usually documented separately in dictionaries and most translations will have a convention to denote each usage. [1] has a good example of how dense these sorts of notational conventions can get. [0] http://oracc.museum.upenn.edu/rinap/Q003230 http://oracc.museum.upenn.edu/rinap/Q003230 [1] https://brill.com/display/book/9789004417564/front-10.xml https://brill.com/display/book/9789004417564/front-10.xml
- thechao 3y ago
  Right? > 51. ki-ma a-tar-tim-ma ul-tu lib3-bi_a-sza3-mesz_ ab-tuq-ma Is translated as: > 51. Like the Apsû, it rose up from the heart of the earth. But 'Apsu' isn't a reading in any of those words?
  - deleted 3y ago
    [deleted]
flenserboy 3y ago
Not only double-check, but probably triple- or quadruple-. There's too much chance of hallucination to trust one or two instances; I think this could be done, but would require a number of disconnected instances doing the translations, with the resulting documents compared with each other to see a) which ones are more prone to making things up, b) if they hallucinate in similar ways, and c) where they agree, which would allow further study to find out what may have been in their training that allowed similar results to be output.
- sp332 3y ago
  I would try round-tripping it at least. But I would have to know a little more Akkadian to tell if it's coming up with synonyms or completely different words.
- earthboundkid 3y ago
  Last week, a story in praise of ChatGPT ended with a haiku that has the wrong syllable count. Why check when it sounds good?
  - gattilorenz 3y ago
    Do you have a link?
    - earthboundkid 3y ago
      https://nadh.in/blog/this-time-it-feels-different/ https://nadh.in/blog/this-time-it-feels-different/ No shade on the author. It just seems like an apt metaphor for how we get blinded by LLMs.
  - sp332 3y ago
    ChatGPT is the only LLM I have seen that can count syllables at all. It seems to be a gap in the way most of them are trained, since they only get text and never “hear” words pronounced.

by sp332 3y ago

AlotOfReading 3y ago

It is. Here's a professional translation for this [0]: ...they made bricks. I razed that small palace in its entirety, took a large area from the fields for an addition, and added (it) to it (the palace). I laid its foundations with limestone, strong stone from the mountains, and raised the terrace. I summoned the kings of Ḫatti and Across the River (Syria-Palestine) Baʾalu, king of Tyre, Manasseh, king of Judah, Qaʾuš-gabri, king of Edom, Muṣurī, king of Moab, Ṣil-Bēl, king of Gaza, Mitinti, king of Ashkelon, ... Even if the translation wasn't very wrong, I'd be incredibly skeptical of this because Akkadian and Sumerian share a lot of words that are written the same way, but have very different meanings (which can even be mixed within a document!). They're usually documented separately in dictionaries and most translations will have a convention to denote each usage. [1] has a good example of how dense these sorts of notational conventions can get. [0] http://oracc.museum.upenn.edu/rinap/Q003230 http://oracc.museum.upenn.edu/rinap/Q003230 [1] https://brill.com/display/book/9789004417564/front-10.xml https://brill.com/display/book/9789004417564/front-10.xml

thechao 3y ago
Right? > 51. ki-ma a-tar-tim-ma ul-tu lib3-bi_a-sza3-mesz_ ab-tuq-ma Is translated as: > 51. Like the Apsû, it rose up from the heart of the earth. But 'Apsu' isn't a reading in any of those words?
- deleted 3y ago
  [deleted]

flenserboy 3y ago

Not only double-check, but probably triple- or quadruple-. There's too much chance of hallucination to trust one or two instances; I think this could be done, but would require a number of disconnected instances doing the translations, with the resulting documents compared with each other to see a) which ones are more prone to making things up, b) if they hallucinate in similar ways, and c) where they agree, which would allow further study to find out what may have been in their training that allowed similar results to be output.

sp332 3y ago
I would try round-tripping it at least. But I would have to know a little more Akkadian to tell if it's coming up with synonyms or completely different words.
earthboundkid 3y ago
Last week, a story in praise of ChatGPT ended with a haiku that has the wrong syllable count. Why check when it sounds good?
- gattilorenz 3y ago
  Do you have a link?
  - earthboundkid 3y ago
    https://nadh.in/blog/this-time-it-feels-different/ https://nadh.in/blog/this-time-it-feels-different/ No shade on the author. It just seems like an apt metaphor for how we get blinded by LLMs.
- sp332 3y ago
  ChatGPT is the only LLM I have seen that can count syllables at all. It seems to be a gap in the way most of them are trained, since they only get text and never “hear” words pronounced.