5 ms·

I use Raku for production. It's great for orchestration, or writing a quick server. Text manipulation and processing is amazing. I'm shocked it hasn't gotten mo

by mempko 2y ago

I use Raku for production. It's great for orchestration, or writing a quick server. Text manipulation and processing is amazing. I'm shocked it hasn't gotten more adoption in the LLM space. You would think someone would make a really great library to create AI Agents using Raku.

klibertp 2y ago
The community is tiny. The fact that it was called "Perl6" for 20 years (Raku is from after 2019 I think) absolutely doesn't help in anything, the least in adoption. The historical churn in terms of VM (experimental in Haskell, dead Parrot, JVM, current MoarVM), the lack of planned features (macros!) for long years, and the slow death of Perl over a similar time frame - all that (and more) means not many people use it, even if they happen to know about it. On the other hand, there's a similarly tiny - or smaller - community around Glamorous Toolkit (a Smalltalk distribution) that managed to produce one of the most hackable and rich LLM clients out there: https://github.com/feenkcom/gt4llm https://github.com/feenkcom/gt4llm https://gtoolkit.com/ https://gtoolkit.com/ Here, though, LLMs are so close a match for the community interests that it just had to happen...
- librasteve 2y ago
  rakulang reddit has 1.5k members ... it's not Python or Rust yet ... but the community is very open and friendly and its great to rub shoulders with such knowledgable and committed folks
  - klibertp 2y ago
    > but the community is very open and friendly Yes, it is - you'll see I said precisely that in another comment here. :) However, I will not yield in that it's tiny. Emacs sub on Reddit has 63k users, half of whom are Emacs Lisp programmers (by necessity). That's 20x more community members than Raku, and we're talking about a) a Lisp; and b) one bundled as an extension language for a text editor... Raku currently has a community comparable to Smalltalk, Racket, or Haxe - enough to keep the lights on, but not much more than that. Similarly to them, Raku doesn't have any strategy for breaking into the mainstream. In effect, these communities don't grow over time. There's an "old guard" - a few dozen people - who are passionate about (and invested in) the project, but the retention of other users rarely exceeds a year or two. I like Raku as a language, but the lack of hands shows. Imagine my surprise when a partially applied sub (`.assuming`) turned out to be 1000x (really) slower than an equivalent block. It's a known issue, just nobody had the time to work on optimizing it. RakuAST work is still ongoing 5 years later. There are other rough edges and unimplemented features, all still there because there is not enough contributors. In any case: Raku is great, and its community is warm and friendly. At the same time, the community is tiny and Raku adoption most likely won't shoot up anytime soon. That doesn't mean you shouldn't use it, but you should be aware of the situation so that you can decide if it offers a suitable trade-off for you.
    - librasteve 2y ago
      I'm glad you like raku and I won't split hairs over small vs. tiny. Since you mention it, I would say that raku is yet to find its killer app. I think for general purpose languages in general, it is hard to predict this zeitgeist - you know Ruby was around for 10+ years before the famous 15 mins Rails video. Another effective strategy for breaking into the mainstream is to have the weight of a big/influential corporation behind you - Java(Sun), JS(Netscape), Go(oogle), Rust(AWS), F#(MS), Swift(Apple) ... that doesn't apply to raku either. I for one would be happy to have a strategy for this - and would welcome your thoughts on what this should contain. In the meantime, I guess the hope for breaking into the mainstream is to do a solid job with language development. Recently the focus has been on Raku AST (which is a forerunner to Macros). The number of passing test-files with the new RakuAST grammar are now 142/153 (make test) and 1158/1355 (make spectest). In addition to this, the community has actively taken over support for the raku IDE which was kindly donated by Edument ... Comma Plugin 2.0 is now available (IntelliJ IDEA plugin) along with several other tooling options. Yes, there have been some (too many!) historic missteps - but the rakudo compiler / moarvm (runtime) combination has been the settled path since 2015. I'm sorry to hear that `.assuming` was 1000x slower codepath for you and that issue was not fixed in a timely way. I choose to use raku in my business for small projects and throw away scripts, data munging and notebook work where the power of the language and ease of quickly getting to working code saves time. While I think a medium-large deployment to make a productised service is viable, it may still have to watch for and step around some gotchas like this. I do contend that the raku community is sufficiently large and committed to deliver continuous improvement overall despite the high ambition and large surface area of the codebase ... perhaps latest GH stats would be a fairer metric (and I have not compared to LISP EMACS ;-) )... https://github.com/rakudo/rakudo/pulse/monthly https://github.com/rakudo/rakudo/pulse/monthly... Excluding merges, 11 authors have pushed 146 commits to main and 258 commits to all branches.
      - klibertp 2y ago
        > Since you mention it, I would say that raku is yet to find its killer app. [...] I for one would be happy to have a strategy for this - and would welcome your thoughts on what this should contain. Please take a look at Glamorous Toolkit (https://gtoolkit.com/ https://gtoolkit.com/). Whether that's warranted or not, the authors claim to have invented a new paradigm of working with software systems. GT is marketed as a "moldable development environment" more than as a "Smalltalk dialect with dynamic, customizable UI." Would finding a catchy phrase describing Raku programming help? I think it could. Or maybe there's already one that I'm not aware of? Even better would be to find something that Raku is unusually well-suited for, better than all, save for just 1 or 2 contenders. Advertising Raku through that could have dramatic effects. However, this is harder to achieve with a general-purpose language. Fortunately, Raku doesn't lack in interesting features and I'm sure there are (still undiscovered or not publicized enough) domains where a particular combination of Raku's features make it one of the best tools for a job. Once you manage to find a few such domains, it's only a matter of luck: at some point, any of these can become popular, elevating the language along with the use case. However, for luck to work, you need to have those domains thoroughly explored - the famous Rails video wouldn't exist without Rails. > Another effective strategy for breaking into the mainstream is to have the weight of a big/influential corporation behind you [...] that doesn't apply to raku either. You might try for corporate sponsorship or government grants. Both ways I know nothing about - I just notice that some work on Haxe is financed through industry grants, while some work on OCaml is paid for by the EU. In general, sustaining a project as large as a programming language is far outside of my expertise; I won't be of much help on that front :( > In the meantime, I guess the hope for breaking into the mainstream is to do a solid job with language development. Yes, that is a good strategy :) As mentioned above, getting into the spotlight and capitalizing on it is largely a matter of luck, and to make use of good luck, you first need a solid project. > Recently the focus has been on Raku AST (which is a forerunner to Macros). That's good - Raku has an infinity of ways for runtime metaprogramming, but almost none for compile-time metaprogramming. Standardized AST and macro support will be a very welcome addition to the language. This might be a good time to mention: are you familiar with how Elixir is developed? I'm not too close to that community, but it looks like the Elixir folks split the work between the language proper and a flagship project of the year (roughly). Those flagships are projects that, ideally, make use of Elixir's unique features to build "best in class" solution. I think Ecto - data persistence library interfacing with RDBMSes - was the first such project. I don't remember the details, but there was a project for IoT (Nerves?) and later Numeric Elixir (numpy equivalent?) - and some other things in between. Phoenix and LiveView, too. I think that's a pretty clever marketing strategy - everybody knows what an ORM (for example) is, so if you write a headline like "Ecto - the best non-ORM you never heard of" you have a chance to get an eyeball or two. > In addition to this, the community has actively taken over support for the raku IDE which was kindly donated by Edument ... Comma Plugin 2.0 is now available (IntelliJ IDEA plugin) along with several other tooling options. I didn't know, thanks for mentioning it. I'll definitely check it out. > Yes, there have been some (too many!) historic missteps - but the rakudo compiler / moarvm (runtime) combination has been the settled path since 2015. Agreed :) > I'm sorry to hear that `.assuming` was 1000x slower codepath for you and that issue was not fixed in a timely way. No, no, it wasn't that big of a deal at all! It was just an example of a "rough edge" that I think was left there because of a lack of manpower. > I choose to use raku in my business for small projects I don't have the luxury to use Raku in my day job, but I use it for small personal projects, mostly related to parsing things, and for general-purpose scripting. > I do contend that the raku community is sufficiently large and committed to deliver continuous improvement overall despite the high ambition and large surface area of the codebase Agreed - I'm sorry if it sounded too negatively, but that's what I meant by "keeping the lights on". Yes - Raku is moving in the right direction at a noticeable (non-glacial) pace. From the looks of it, the compiler and the VM are solid pieces of engineering and they are improving. However, without a concurrent effort in marketing the language, preferably with some "killer apps" already in place, "just" doing a good job implementing the language might not be enough for Raku to succeed. That's my main worry :( --- One thing that didn't fit above. Are there plans to improve static type checking in Raku? Raku looks gradually-typed, but actually isn't - most of the type checks happen on runtime, at least in my experience. This is very similar to Dylan language. Is there any work being done on strengthtening the type system and adding type inference, occurence typing, and (maybe) refinement types? It should be possible - I read a paper about adding gradual typing to Dylan, which also is based on multi-methods and multiple inheritance. So I think there's a lot of type-checking the compiler could (try to) do. IMO, going in this direction would help Raku codebases scale to larger projects. EDIT: if you'd like to continue the discussion, please shoot me an email (in my profile).
librasteve 2y ago
please try googling "raku llm" ... there is a lot of excellent thought leadership work from folks like antononcube
antononcube 2y ago
## Clarification Can you clarify what do you mean by: > You would think someone would make a really great library to create AI Agents using Raku. Maybe, you mean using Raku to make: 1. Standalone, web based LLM agents 2. Plug-ins for ChatGPT (or other LLM providers) 3. LLM Retrieval Augmented Generation (RAG) functionalities, or services, or objects LLM RAG (point 3) is supported in Raku, see: "Raku RAG demo", https://youtu.be/JHO2Wk1b-Og https://youtu.be/JHO2Wk1b-Og . Points 1 and 2 should be Raku-addressed sooner or later, but Raku provides nice alternatives to them. (See below.) ## LLMs at your fingertips I would say, with the Raku LLM packages one can create "AI agents" on the spot: - Programmatic ways to configure and execute LLM functions and chat objects. - Using easy to compose LLM functions and easy to access prompts. - Via Jupyter based notebooks with LLM-ready magic cells, aka "chatbooks." In one chatbook multiple, global chats can be held with multiple LLM personas. Also, each of the LLM-service access packages provides a Command Line Interface (CLI) script. That is another quick, "on the spot" type of access to AI. ## References Please see these Raku LLM packages: https://raku.land/?q=LLM https://raku.land/?q=LLM

by mempko 2y ago

klibertp 2y ago

The community is tiny. The fact that it was called "Perl6" for 20 years (Raku is from after 2019 I think) absolutely doesn't help in anything, the least in adoption. The historical churn in terms of VM (experimental in Haskell, dead Parrot, JVM, current MoarVM), the lack of planned features (macros!) for long years, and the slow death of Perl over a similar time frame - all that (and more) means not many people use it, even if they happen to know about it. On the other hand, there's a similarly tiny - or smaller - community around Glamorous Toolkit (a Smalltalk distribution) that managed to produce one of the most hackable and rich LLM clients out there: https://github.com/feenkcom/gt4llm https://github.com/feenkcom/gt4llm https://gtoolkit.com/ https://gtoolkit.com/ Here, though, LLMs are so close a match for the community interests that it just had to happen...

librasteve 2y ago
rakulang reddit has 1.5k members ... it's not Python or Rust yet ... but the community is very open and friendly and its great to rub shoulders with such knowledgable and committed folks
- klibertp 2y ago
  > but the community is very open and friendly Yes, it is - you'll see I said precisely that in another comment here. :) However, I will not yield in that it's tiny. Emacs sub on Reddit has 63k users, half of whom are Emacs Lisp programmers (by necessity). That's 20x more community members than Raku, and we're talking about a) a Lisp; and b) one bundled as an extension language for a text editor... Raku currently has a community comparable to Smalltalk, Racket, or Haxe - enough to keep the lights on, but not much more than that. Similarly to them, Raku doesn't have any strategy for breaking into the mainstream. In effect, these communities don't grow over time. There's an "old guard" - a few dozen people - who are passionate about (and invested in) the project, but the retention of other users rarely exceeds a year or two. I like Raku as a language, but the lack of hands shows. Imagine my surprise when a partially applied sub (`.assuming`) turned out to be 1000x (really) slower than an equivalent block. It's a known issue, just nobody had the time to work on optimizing it. RakuAST work is still ongoing 5 years later. There are other rough edges and unimplemented features, all still there because there is not enough contributors. In any case: Raku is great, and its community is warm and friendly. At the same time, the community is tiny and Raku adoption most likely won't shoot up anytime soon. That doesn't mean you shouldn't use it, but you should be aware of the situation so that you can decide if it offers a suitable trade-off for you.
  - librasteve 2y ago
    I'm glad you like raku and I won't split hairs over small vs. tiny. Since you mention it, I would say that raku is yet to find its killer app. I think for general purpose languages in general, it is hard to predict this zeitgeist - you know Ruby was around for 10+ years before the famous 15 mins Rails video. Another effective strategy for breaking into the mainstream is to have the weight of a big/influential corporation behind you - Java(Sun), JS(Netscape), Go(oogle), Rust(AWS), F#(MS), Swift(Apple) ... that doesn't apply to raku either. I for one would be happy to have a strategy for this - and would welcome your thoughts on what this should contain. In the meantime, I guess the hope for breaking into the mainstream is to do a solid job with language development. Recently the focus has been on Raku AST (which is a forerunner to Macros). The number of passing test-files with the new RakuAST grammar are now 142/153 (make test) and 1158/1355 (make spectest). In addition to this, the community has actively taken over support for the raku IDE which was kindly donated by Edument ... Comma Plugin 2.0 is now available (IntelliJ IDEA plugin) along with several other tooling options. Yes, there have been some (too many!) historic missteps - but the rakudo compiler / moarvm (runtime) combination has been the settled path since 2015. I'm sorry to hear that `.assuming` was 1000x slower codepath for you and that issue was not fixed in a timely way. I choose to use raku in my business for small projects and throw away scripts, data munging and notebook work where the power of the language and ease of quickly getting to working code saves time. While I think a medium-large deployment to make a productised service is viable, it may still have to watch for and step around some gotchas like this. I do contend that the raku community is sufficiently large and committed to deliver continuous improvement overall despite the high ambition and large surface area of the codebase ... perhaps latest GH stats would be a fairer metric (and I have not compared to LISP EMACS ;-) )... https://github.com/rakudo/rakudo/pulse/monthly https://github.com/rakudo/rakudo/pulse/monthly... Excluding merges, 11 authors have pushed 146 commits to main and 258 commits to all branches.
    - klibertp 2y ago
      > Since you mention it, I would say that raku is yet to find its killer app. [...] I for one would be happy to have a strategy for this - and would welcome your thoughts on what this should contain. Please take a look at Glamorous Toolkit (https://gtoolkit.com/ https://gtoolkit.com/). Whether that's warranted or not, the authors claim to have invented a new paradigm of working with software systems. GT is marketed as a "moldable development environment" more than as a "Smalltalk dialect with dynamic, customizable UI." Would finding a catchy phrase describing Raku programming help? I think it could. Or maybe there's already one that I'm not aware of? Even better would be to find something that Raku is unusually well-suited for, better than all, save for just 1 or 2 contenders. Advertising Raku through that could have dramatic effects. However, this is harder to achieve with a general-purpose language. Fortunately, Raku doesn't lack in interesting features and I'm sure there are (still undiscovered or not publicized enough) domains where a particular combination of Raku's features make it one of the best tools for a job. Once you manage to find a few such domains, it's only a matter of luck: at some point, any of these can become popular, elevating the language along with the use case. However, for luck to work, you need to have those domains thoroughly explored - the famous Rails video wouldn't exist without Rails. > Another effective strategy for breaking into the mainstream is to have the weight of a big/influential corporation behind you [...] that doesn't apply to raku either. You might try for corporate sponsorship or government grants. Both ways I know nothing about - I just notice that some work on Haxe is financed through industry grants, while some work on OCaml is paid for by the EU. In general, sustaining a project as large as a programming language is far outside of my expertise; I won't be of much help on that front :( > In the meantime, I guess the hope for breaking into the mainstream is to do a solid job with language development. Yes, that is a good strategy :) As mentioned above, getting into the spotlight and capitalizing on it is largely a matter of luck, and to make use of good luck, you first need a solid project. > Recently the focus has been on Raku AST (which is a forerunner to Macros). That's good - Raku has an infinity of ways for runtime metaprogramming, but almost none for compile-time metaprogramming. Standardized AST and macro support will be a very welcome addition to the language. This might be a good time to mention: are you familiar with how Elixir is developed? I'm not too close to that community, but it looks like the Elixir folks split the work between the language proper and a flagship project of the year (roughly). Those flagships are projects that, ideally, make use of Elixir's unique features to build "best in class" solution. I think Ecto - data persistence library interfacing with RDBMSes - was the first such project. I don't remember the details, but there was a project for IoT (Nerves?) and later Numeric Elixir (numpy equivalent?) - and some other things in between. Phoenix and LiveView, too. I think that's a pretty clever marketing strategy - everybody knows what an ORM (for example) is, so if you write a headline like "Ecto - the best non-ORM you never heard of" you have a chance to get an eyeball or two. > In addition to this, the community has actively taken over support for the raku IDE which was kindly donated by Edument ... Comma Plugin 2.0 is now available (IntelliJ IDEA plugin) along with several other tooling options. I didn't know, thanks for mentioning it. I'll definitely check it out. > Yes, there have been some (too many!) historic missteps - but the rakudo compiler / moarvm (runtime) combination has been the settled path since 2015. Agreed :) > I'm sorry to hear that `.assuming` was 1000x slower codepath for you and that issue was not fixed in a timely way. No, no, it wasn't that big of a deal at all! It was just an example of a "rough edge" that I think was left there because of a lack of manpower. > I choose to use raku in my business for small projects I don't have the luxury to use Raku in my day job, but I use it for small personal projects, mostly related to parsing things, and for general-purpose scripting. > I do contend that the raku community is sufficiently large and committed to deliver continuous improvement overall despite the high ambition and large surface area of the codebase Agreed - I'm sorry if it sounded too negatively, but that's what I meant by "keeping the lights on". Yes - Raku is moving in the right direction at a noticeable (non-glacial) pace. From the looks of it, the compiler and the VM are solid pieces of engineering and they are improving. However, without a concurrent effort in marketing the language, preferably with some "killer apps" already in place, "just" doing a good job implementing the language might not be enough for Raku to succeed. That's my main worry :( --- One thing that didn't fit above. Are there plans to improve static type checking in Raku? Raku looks gradually-typed, but actually isn't - most of the type checks happen on runtime, at least in my experience. This is very similar to Dylan language. Is there any work being done on strengthtening the type system and adding type inference, occurence typing, and (maybe) refinement types? It should be possible - I read a paper about adding gradual typing to Dylan, which also is based on multi-methods and multiple inheritance. So I think there's a lot of type-checking the compiler could (try to) do. IMO, going in this direction would help Raku codebases scale to larger projects. EDIT: if you'd like to continue the discussion, please shoot me an email (in my profile).

librasteve 2y ago

please try googling "raku llm" ... there is a lot of excellent thought leadership work from folks like antononcube

antononcube 2y ago

## Clarification Can you clarify what do you mean by: > You would think someone would make a really great library to create AI Agents using Raku. Maybe, you mean using Raku to make: 1. Standalone, web based LLM agents 2. Plug-ins for ChatGPT (or other LLM providers) 3. LLM Retrieval Augmented Generation (RAG) functionalities, or services, or objects LLM RAG (point 3) is supported in Raku, see: "Raku RAG demo", https://youtu.be/JHO2Wk1b-Og https://youtu.be/JHO2Wk1b-Og . Points 1 and 2 should be Raku-addressed sooner or later, but Raku provides nice alternatives to them. (See below.) ## LLMs at your fingertips I would say, with the Raku LLM packages one can create "AI agents" on the spot: - Programmatic ways to configure and execute LLM functions and chat objects. - Using easy to compose LLM functions and easy to access prompts. - Via Jupyter based notebooks with LLM-ready magic cells, aka "chatbooks." In one chatbook multiple, global chats can be held with multiple LLM personas. Also, each of the LLM-service access packages provides a Command Line Interface (CLI) script. That is another quick, "on the spot" type of access to AI. ## References Please see these Raku LLM packages: https://raku.land/?q=LLM https://raku.land/?q=LLM