4 ms·

Has anyone used Tcl for LLM prompt control?

vessenes 3y ago
I like this idea. TCL's major strength is combining up a practical lispy-ness with 'everything is a string'. I bet that would fit well with LLM-land prompting, especially in a hacking sort of environment. I haven't coded TCL in something like 20 years; at the time Unicode support in TCL was non-existent; a quick review of modern TCL forums looks like UTF-8 support is not wart-free, and UTF-16 support is not on the table. I used TCL on top of AOLServer, and at its best, it really did give that "you have superpowers" feeling Lisp can give an engineer. I bet it would make a very comfortable environment for experimenting. So much of modern LLM experimentation could be seen as creating DSLs for different tasks: TCL excels at this.
- andrewshadura 3y ago
  Tcl has supported Unicode and UTF-8 for ages, but to learn that you can just check its excellent documentation, no need to browse forums :)
  - vessenes 3y ago
    https://wiki.tcl-lang.org/page/Unicode+and+UTF-8 https://wiki.tcl-lang.org/page/Unicode+and+UTF-8 This is what I referred to, which makes it look like emoji support is spotty. But that's all I read!
    - spijdar 3y ago
      It's been a while, but my recollection is that the Tcl interpreter stores strings as fixed-width code points, and uses 16 bit characters by default, so it's effectively UCS-2 (hence being limited to the first 65536 characters). This is configurable, though -- I recall it was a preprocessor define in the C code, and you can change it to 32 bits per character, making it UTF-32. Of course, this won't help a Tcl script run somewhere else, but it means you _can_ build a Tcl interpreter that transparently works with "all unicode", at the cost of doubling memory consumption for every string. All this is concerning the internal string format, not what it accepts as input/output.
    - cmacleod4 3y ago
      The next major release Tcl 9.0 will support the full expanded unicode range, so you can properly manipulate strings containing Sanskrit, Arabic and Pile-of-Poo! Beta release coming "Real Soon Now" :-)

I like this idea. TCL's major strength is combining up a practical lispy-ness with 'everything is a string'. I bet that would fit well with LLM-land prompting, especially in a hacking sort of environment. I haven't coded TCL in something like 20 years; at the time Unicode support in TCL was non-existent; a quick review of modern TCL forums looks like UTF-8 support is not wart-free, and UTF-16 support is not on the table. I used TCL on top of AOLServer, and at its best, it really did give that "you have superpowers" feeling Lisp can give an engineer. I bet it would make a very comfortable environment for experimenting. So much of modern LLM experimentation could be seen as creating DSLs for different tasks: TCL excels at this.

andrewshadura 3y ago
Tcl has supported Unicode and UTF-8 for ages, but to learn that you can just check its excellent documentation, no need to browse forums :)
- vessenes 3y ago
  https://wiki.tcl-lang.org/page/Unicode+and+UTF-8 https://wiki.tcl-lang.org/page/Unicode+and+UTF-8 This is what I referred to, which makes it look like emoji support is spotty. But that's all I read!
  - spijdar 3y ago
    It's been a while, but my recollection is that the Tcl interpreter stores strings as fixed-width code points, and uses 16 bit characters by default, so it's effectively UCS-2 (hence being limited to the first 65536 characters). This is configurable, though -- I recall it was a preprocessor define in the C code, and you can change it to 32 bits per character, making it UTF-32. Of course, this won't help a Tcl script run somewhere else, but it means you _can_ build a Tcl interpreter that transparently works with "all unicode", at the cost of doubling memory consumption for every string. All this is concerning the internal string format, not what it accepts as input/output.
  - cmacleod4 3y ago
    The next major release Tcl 9.0 will support the full expanded unicode range, so you can properly manipulate strings containing Sanskrit, Arabic and Pile-of-Poo! Beta release coming "Real Soon Now" :-)