4 ms·

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

gizmo686 1y ago
Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream. From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.
- cubefox 1y ago
  > Hebrew is still written sequentially Everything is written sequentially in the sense that the character that is written first can only be followed by the character that is written next. In this sense writing non-sequentially is logically impossible.
  - goatlover 1y ago
    Multiple characters can be written at once, they can also be done in reverse or out of order.
    - cubefox 1y ago
      No no, the second character you write must always be temporally preceded by the character you wrote first. Otherwise the second wouldn't have been the second, but the first, and moreover, the first would have been the second, which it wasn't.
      - dotancohen 1y ago
        I encourage you to find some place that still uses a Hebrew typewriter. When they have to type numbers, they'll type the number in backwards. And an old Hebrew encoding also encoded characters in reverse order.
        short_sells_poo 1y ago
        I think parent just means that "backwards" is a relative term. Your backwards is someone else's "forward". For someone who is used to reading Hebrew, they would be used to reading right to left and this would seem completely natural, no? Basically, the numbers 1234 and 4321 are identical assuming one is written left to right and the other is right to left. Then it's just a convention which way you are used to reading. I know nothing of Old (or New) Hebrew unfortunately so I may be completely off base.
        gowld 1y ago
        No, because Hebrew words are read right-to-left in Hebrew letters, but numbers are read left-to-right in Arabaic numerals. The direction of reading switches mid-sentence, but typewriters only type in one direction.
        cubefox 1y ago
        Arguably Arabic numbers must always be read right-to-left, even in English, because the least significant digits can be read in order, while the value of the most significant digit depends on the number of less significant digits to the right. So in Hebrew the general reading direction actually fits Arabic numbers better.
        dotancohen 1y ago
        And the Arabs actually say the numbers from right to left. It's "one and fifty", not "fifty one".
        1y ago
      - Someone 1y ago
        You could write multiple characters simultaneously. CRTs sort-of did that, for example, starting characters with ascenders before those without and finishing the characters without descenders before those with descenders. So, in the word “gif”, they would start writing the “f” first and finish writing the “i” first (just before writing the last part of the “f”. For “if”, writing the “f” would start before writing the “i” started and finish after writing the “i” finished. In traditional printing “writing” can happen simultaneously for an entire page, but colour printing can make things more complex.
  - dotancohen 1y ago
    An older Hebrew encoding actually encoded the last character first, then the penultimate character, then the character preceding that, etc. Exercise to the reader to guess how line breaks, text wrapping, and search algorithms worked.
  - deleted 1y ago
    [deleted]
- Timwi 1y ago
  I know I'm being pedantic, but I just want to point out that even U+200E/U+200F are generally not needed. If you put a Hebrew word in the middle of an English sentence, it displays correctly all by itself. This is due to the Unicode bidirectional algorithm, which defines a super sensible default behavior. You only need the RTL control characters in weird circumstances, perhaps ones involving punctuation marks or unusual uses of special characters.
fer 1y ago
>Seems like it could easily be training data set size as well. I'm convinced that's the case. On any major LLM I can carpet bomb Java/Python boilerplate without issue. For Rust, at least last time I checked, it comes up with non-existing traits, more frequent hallucinations and general struggle to use the context effectively. In agent mode it turns into a first fight with the compiler, often ending in credit destroying loops. And don't get me started when using it for Nix... So not surprised about something with orders of magnitude smaller public corpus.
- dlahoda 1y ago
  i tried gemini, openai, copilot, claude on reasonably big rust project. claude worked well to fix use, clippy, renames, refactorings, ci. i used highest cost claude with custom context per crate. never was able to get it write new code well. for nix, i is nice template engine to start or search. did not tried big nix changes.
- dotancohen 1y ago
  I realized this too, and it led me to the conclusion that LLMs really can't program. I did some experiments to find what a programming language would look like, instead of e.g. python, if it were designed to be written and edited by an LLM. It turns out that it's extremely verbose, especially in variable names, function names, class names, etc. Actually, it turned out that classes were very redundant. But the real insight was that LLMs are great at naming things, and performing small operations on the little things they named. They're really not good at any logic that they can't copy paste from something they found on the web.
  - weird-eye-issue 1y ago
    > I did some experiments to find what a programming language would look like, instead of e.g. python, if it were designed to be written and edited by an LLM. Did your experiment consist of asking an LLM to design a programming language for itself?
    - dotancohen 1y ago
44za12 1y ago
That’s what I thought. Lack of training data might be a reason.
MonkeyClub 1y ago
> (or hebrew) W.r.t. natural languages, TFA clarifies it a bit: > And it’s not the same as translation to Arabic or Hebrew; direction here refers to the temporal order in which the tokens are produced; even for right-to-left languages, the order in which the tokens get produced remains unchanged; rather, a thin display layer handles the visual presentation.

Seems like it could easily be training data set size as well. I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

gizmo686 1y ago

Hebrew is still written sequentially in Unicode. The right-to-left aspect there is simply about how the characters get displayed. On mixed documents, there is U+200E and U+200F to change the text direction mid stream. From the perspective of a LLM learning from Unicode, this would appear as a delimeter that needs to be inserted on language direction boundaries; but everything else should work the same.

cubefox 1y ago
> Hebrew is still written sequentially Everything is written sequentially in the sense that the character that is written first can only be followed by the character that is written next. In this sense writing non-sequentially is logically impossible.
- goatlover 1y ago
  Multiple characters can be written at once, they can also be done in reverse or out of order.
  - cubefox 1y ago
    No no, the second character you write must always be temporally preceded by the character you wrote first. Otherwise the second wouldn't have been the second, but the first, and moreover, the first would have been the second, which it wasn't.
    - dotancohen 1y ago
      I encourage you to find some place that still uses a Hebrew typewriter. When they have to type numbers, they'll type the number in backwards. And an old Hebrew encoding also encoded characters in reverse order.
      - short_sells_poo 1y ago
        I think parent just means that "backwards" is a relative term. Your backwards is someone else's "forward". For someone who is used to reading Hebrew, they would be used to reading right to left and this would seem completely natural, no? Basically, the numbers 1234 and 4321 are identical assuming one is written left to right and the other is right to left. Then it's just a convention which way you are used to reading. I know nothing of Old (or New) Hebrew unfortunately so I may be completely off base.
        gowld 1y ago
        No, because Hebrew words are read right-to-left in Hebrew letters, but numbers are read left-to-right in Arabaic numerals. The direction of reading switches mid-sentence, but typewriters only type in one direction.
        cubefox 1y ago
        Arguably Arabic numbers must always be read right-to-left, even in English, because the least significant digits can be read in order, while the value of the most significant digit depends on the number of less significant digits to the right. So in Hebrew the general reading direction actually fits Arabic numbers better.
        dotancohen 1y ago
        And the Arabs actually say the numbers from right to left. It's "one and fifty", not "fifty one".
        1y ago
    - Someone 1y ago
      You could write multiple characters simultaneously. CRTs sort-of did that, for example, starting characters with ascenders before those without and finishing the characters without descenders before those with descenders. So, in the word “gif”, they would start writing the “f” first and finish writing the “i” first (just before writing the last part of the “f”. For “if”, writing the “f” would start before writing the “i” started and finish after writing the “i” finished. In traditional printing “writing” can happen simultaneously for an entire page, but colour printing can make things more complex.
- dotancohen 1y ago
  An older Hebrew encoding actually encoded the last character first, then the penultimate character, then the character preceding that, etc. Exercise to the reader to guess how line breaks, text wrapping, and search algorithms worked.
- deleted 1y ago
  [deleted]
Timwi 1y ago
I know I'm being pedantic, but I just want to point out that even U+200E/U+200F are generally not needed. If you put a Hebrew word in the middle of an English sentence, it displays correctly all by itself. This is due to the Unicode bidirectional algorithm, which defines a super sensible default behavior. You only need the RTL control characters in weird circumstances, perhaps ones involving punctuation marks or unusual uses of special characters.

fer 1y ago

>Seems like it could easily be training data set size as well. I'm convinced that's the case. On any major LLM I can carpet bomb Java/Python boilerplate without issue. For Rust, at least last time I checked, it comes up with non-existing traits, more frequent hallucinations and general struggle to use the context effectively. In agent mode it turns into a first fight with the compiler, often ending in credit destroying loops. And don't get me started when using it for Nix... So not surprised about something with orders of magnitude smaller public corpus.

dlahoda 1y ago
i tried gemini, openai, copilot, claude on reasonably big rust project. claude worked well to fix use, clippy, renames, refactorings, ci. i used highest cost claude with custom context per crate. never was able to get it write new code well. for nix, i is nice template engine to start or search. did not tried big nix changes.
dotancohen 1y ago
I realized this too, and it led me to the conclusion that LLMs really can't program. I did some experiments to find what a programming language would look like, instead of e.g. python, if it were designed to be written and edited by an LLM. It turns out that it's extremely verbose, especially in variable names, function names, class names, etc. Actually, it turned out that classes were very redundant. But the real insight was that LLMs are great at naming things, and performing small operations on the little things they named. They're really not good at any logic that they can't copy paste from something they found on the web.
- weird-eye-issue 1y ago
  > I did some experiments to find what a programming language would look like, instead of e.g. python, if it were designed to be written and edited by an LLM. Did your experiment consist of asking an LLM to design a programming language for itself?
  - dotancohen 1y ago

44za12 1y ago

That’s what I thought. Lack of training data might be a reason.

MonkeyClub 1y ago

> (or hebrew) W.r.t. natural languages, TFA clarifies it a bit: > And it’s not the same as translation to Arabic or Hebrew; direction here refers to the temporal order in which the tokens are produced; even for right-to-left languages, the order in which the tokens get produced remains unchanged; rather, a thin display layer handles the visual presentation.