5 ms·

> Rails has started to show its age amid with the current wave of AI-powered applications. It struggles with LLM text streaming, parallel processing in Ruby, an

by btown 2y ago

> Rails has started to show its age amid with the current wave of AI-powered applications. It struggles with LLM text streaming, parallel processing in Ruby, and lacks strong typing for AI coding tools. Despite these constraints, it remains effective. A plug for Django + gevent in this context! You have the Python type system, and while it's inferior to TypeScript's in many ways, it's far more ubiquitous than Ruby's Sorbet. For streaming and any kind of IO-bound parallelism, gevent's monkey-patches cause every blocking operation to become a event-loop yield... so you can stream many concurrent responses at a time, with a simple generator. CPU-bound parallelism doesn't have a great story here, but that's less relevant for web applications - and if you're simultaneously iterating on ML models and a web backend, they'd likely run on separate machines anyways, and you can write both in Python without context-switching as a developer.

cess11 2y ago
Just move to Elixir. Phoenix is Rails-like enough and the platform is excellent for parallelisation, clustering in specific hardware and so on.
- arrowsmith 2y ago
  And if you want to make the move, I know a great resource: http://PhoenixOnRails.com http://PhoenixOnRails.com </shamelessselfplug>
- freedomben 2y ago
  Normally "switch languages" isn't great advice, but in this case I think it's worth considering. I have heard people coming from Django and Rails background describe Elixir as "a love child between python and ruby". Personally I love it
  - aragilar 2y ago
    But does Elixir come with a whole scientific computing ecosystem?
    - aloha2436 2y ago
      Not to the same degree that Python does (then again no other general-purpose language does!), but it does have the start of one and it's fairly cohesive. https://github.com/elixir-nx https://github.com/elixir-nx
      - aragilar 2y ago
        Fortran? R? C? C++? Even Java may occasionally make a good showing here (depending on what you are doing). Having seen... things... unless it's written by people with the right skillset (and with funding and the right environment), that it exists doesn't mean you should use it (and the phrase "it's a trap" comes to mind sadly). https://scicomp.stackexchange.com/a/10923/1437 https://scicomp.stackexchange.com/a/10923/1437 applies (and note I still wouldn't call Julia mainstream yet), so while I'm not saying people shouldn't try, the phrase "don't roll your own crypto" applies just as much to the numeric and scientific computing fields.
    - the_gastropod 2y ago
      I'm not super tuned into the scientific computing ecosystem, so not sure if this is what you mean. But maybe? Elixir's Numerical Elixir projects seem very relevant for scientific computing. Check 'em out: https://github.com/elixir-nx https://github.com/elixir-nx Edit: Hah! aloha2436 beat me to the answer. Sorry for the repetition.
    - goosejuice 2y ago
      Must your statistical computing ecosystem comingle with your web interface?
      - aragilar 2y ago
        You can split it off and have your Python code be an API you call, but now you have at least two languages involved (Python+Elixir, plus JS somewhere, plus the possible mix of C/C++/Fortran/Rust(maybe?)). Given Ruby on Rails was mentioned, just using Django seems similarly like the least risky thing to do (this all assumes you are doing numerical stuff, not just a standard CRUD app).
- seneca 2y ago
  I personally think Elixir is a great language, but the jump from ruby to functional programming is big enough that I'm not sure it's useful general advice.
  - vlunkr 2y ago
    Also, the size of the elixir community and the libraries available is completely dwarfed by rails. Elixir, Phoenix, all the core stuff is really high quality, but in many cases you might doing more work that you could have just pulled from a gem in Ruby. It's unfortunate IMO. It's an underrated language.
    - pythonaut_16 2y ago
      Very much this. I think the community tends to overestimate the ecosystem’s maturity which is one of the big things holding it back, both because it blinds the community to areas that need improvement and leads to bigger shocks when newcomers do unexpectedly run into the rough edges.
    - arrowsmith 2y ago
      This is true, but AI is closing the gap. "Take this Ruby gem and write the equivalent functionality in Elixir" is something that LLMs are very good at.
      - weatherlite 2y ago
        That's over simplifying things ... no, complex gems cannot simply be transformed to Elixir with ChatGPT. You'd have to have an expert in both languages fixing all the bugs.
    - cess11 2y ago
      Do you have some examples?
- paradox460 2y ago
  And the switch is rather easy. I've been writing elixir for nearly 10 years, rails before that, and have overseen the "conversion" of several engineers from one to the other. Generally I'd say any senior rails dev, given the right task, can write decent elixir code on their first day. There are a lot fewer foot guns in elixir and Phoenix, and so other than language ergonomics (a simple rule that doesn't stretch far but works at the beginning is use pipe instead of dot), there's minimal barriers
  - troad 2y ago
    Honest question from someone working on a non-negligible Rails codebase: what would be my gains, were I to switch to Elixir? I've watched Elixir with much interest from afar, I even recently cracked open the book on it, but I feel like my biggest pain points with Ruby are performance and lack of gradual typing (and consequent lack of static analysis, painful refactoring, etc), and it doesn't really seem like Elixir has much to offer on those. What does Elixir solve, that Ruby struggles on?
    - paradox460 2y ago
      Performance is a complex story. Elixir is very good at massive amounts of parallel computing, and uses this to handle each request (or socket) in it's own contained manner, which nets you some simplicity in designing systems and scaling. However it's not very good at single threaded off-the-block performance (but neither is Ruby) Typing is coming, some is already here, and if you're impatient you can use dialyzer to get half way there. But in my experience you need it less than you'd think. Elixir primitives are rather expressive, and due to the functional nature there really isn't any "new" data structure that isn't easy to dive into. And with each function head being easy to pattern match in, you can dictate the shape of data flow through your application much as you'd do in a typed language. The ide story isn't great, but it's getting better. There are a few LSPs out there, and an official one coming soon. And I'd say all of them beat solargraph handily But most of all I'd say that, since it's a bit more strict, elixir saves you and your coworkers from yourselves and each other. There are still several ways to do something, like Ruby, but going out of your way to write very cutesy code that the next programmer will loathe is more difficult. Not impossible, but harder. And with the rock stable concurrency systems in place, a lot of the impetus to come up with clever solutions isn't there
jackbravo 2y ago
If you want something more similar to Next.JS but in the python world, now you have https://fastht.ml/ https://fastht.ml/, which also has a big performance benefit over Django. Hahaha, same as Next.JS over Rails, because it is much more bare bones. But I would say that fasthtml has the advantage of being super easy to integrate more AI libraries from the python world.
- zzzeek 2y ago
  now that was a crazy rabbit hole
simonw 2y ago
Django shouldn't even require gevent - Django's ASGI support has been baking for a few releases now and supports async views which should be well suited to proxying streams from LLMs etc. Relevant: - https://fly.io/django-beats/running-tasks-concurrently-in-django-asynchronous-views/ https://fly.io/django-beats/running-tasks-concurrently-in-dj... - https://blog.pecar.me/django-streaming-responses https://blog.pecar.me/django-streaming-responses (Reminds me I should try that out properly myself.)
- pphysch 2y ago
  then you have to rewrite your whole app to use asyncio keywords and colored ORM methods. A gevent monkey patch, or eventually nogil concurrency makes a lot more practical sense.
  - simonw 2y ago
    You don't have to rewrite your whole app - you can continue using the regular stuff in synchronous view functions, then have a few small async views for your LLM streaming pieces. I've never quite gotten comfortable with gevent patches, but that's more because I don't personally understand them or what their edge cases might be than a commentary on their reliability.
    - pphysch 2y ago
      Last time I checked, simply running Django through the ASGI interface dealt you a baseline perf hit.
      - simonw 2y ago
        It's pretty minimal: https://docs.djangoproject.com/en/5.1/topics/async/#performance https://docs.djangoproject.com/en/5.1/topics/async/#performa... > When running in a mode that does not match the view (e.g. an async view under WSGI, or a traditional sync view under ASGI), Django must emulate the other call style to allow your code to run. This context-switch causes a small performance penalty of around a millisecond.
troad 2y ago
> You have the Python type system, and while it's inferior to TypeScript's in many ways, it's far more ubiquitous than Ruby's Sorbet. I'm a big fan of Ruby, but God I wish it had good, in-line type hinting. Sorbet annotations are too noisy and the whole thing feels very awkwardly bolted on, while RBS' use of external files make it a non-starter.
- lloeki 2y ago
  I for one really like RBS being external files, it keeps the Ruby side of things uncluttered. When I do need types inline I believe it is the editor's job to show them dynamically, e.g via facilities like tooltips, autocompletion, or vim inlay hints and virtual text, which can apply to much more than just signatures near method definitions. Types are much more useful where the code is used than where it is defined. I follow a 1:1 lib/.rb - sig/.rbs convention and have projection+ files to jump from one to the other instantly. And since the syntax of RBS is so close to Ruby I found myself accidentally writing things type-first then using that as a template to write the actual code. Of note, if you crawl soutaro's repo (author of steep) you'll find a prototype of inline RBS. + used by vim projectionist and vscode projection extension
- sankha93 2y ago
  Do you mean Ruby lacks syntactic support for adding type annotations inline in your programs? I am one of the authors of RDL (https://github.com/tupl-tufts/rdl https://github.com/tupl-tufts/rdl) a research project that looked at type systems for Ruby before it became mainstream. We went for strings that looked nice, but were parsed into a type signature. Sorbet, on the other hand, uses Ruby values in a DSL to define types. We were of the impression that many of our core ideas were absorbed by other projects and Sorbet and RBS has pretty much mainstream. What is missing to get usable gradual types in Ruby?
  - pmontra 2y ago
    None of my customers ever asked for type definitions in Ruby (nor in Python.) I'm pretty happy of the choice of hiding types under the carpet of a separate file. I think they made it deliberately because Ruby's core team didn't like type definitions but had to cave to the recent fashion. It will swing back but I think that this is a slow pendulum. Talking about myself I picked Ruby 20 years ago exactly because I didn't have to type types so I'm not a fan of the projects you are working at, but I don't even oppose them. I just wish I'm never forced to define types.

by btown 2y ago

cess11 2y ago

Just move to Elixir. Phoenix is Rails-like enough and the platform is excellent for parallelisation, clustering in specific hardware and so on.

arrowsmith 2y ago
And if you want to make the move, I know a great resource: http://PhoenixOnRails.com http://PhoenixOnRails.com </shamelessselfplug>
freedomben 2y ago
Normally "switch languages" isn't great advice, but in this case I think it's worth considering. I have heard people coming from Django and Rails background describe Elixir as "a love child between python and ruby". Personally I love it
- aragilar 2y ago
  But does Elixir come with a whole scientific computing ecosystem?
  - aloha2436 2y ago
    Not to the same degree that Python does (then again no other general-purpose language does!), but it does have the start of one and it's fairly cohesive. https://github.com/elixir-nx https://github.com/elixir-nx
    - aragilar 2y ago
      Fortran? R? C? C++? Even Java may occasionally make a good showing here (depending on what you are doing). Having seen... things... unless it's written by people with the right skillset (and with funding and the right environment), that it exists doesn't mean you should use it (and the phrase "it's a trap" comes to mind sadly). https://scicomp.stackexchange.com/a/10923/1437 https://scicomp.stackexchange.com/a/10923/1437 applies (and note I still wouldn't call Julia mainstream yet), so while I'm not saying people shouldn't try, the phrase "don't roll your own crypto" applies just as much to the numeric and scientific computing fields.
  - the_gastropod 2y ago
    I'm not super tuned into the scientific computing ecosystem, so not sure if this is what you mean. But maybe? Elixir's Numerical Elixir projects seem very relevant for scientific computing. Check 'em out: https://github.com/elixir-nx https://github.com/elixir-nx Edit: Hah! aloha2436 beat me to the answer. Sorry for the repetition.
  - goosejuice 2y ago
    Must your statistical computing ecosystem comingle with your web interface?
    - aragilar 2y ago
      You can split it off and have your Python code be an API you call, but now you have at least two languages involved (Python+Elixir, plus JS somewhere, plus the possible mix of C/C++/Fortran/Rust(maybe?)). Given Ruby on Rails was mentioned, just using Django seems similarly like the least risky thing to do (this all assumes you are doing numerical stuff, not just a standard CRUD app).
seneca 2y ago
I personally think Elixir is a great language, but the jump from ruby to functional programming is big enough that I'm not sure it's useful general advice.
- vlunkr 2y ago
  Also, the size of the elixir community and the libraries available is completely dwarfed by rails. Elixir, Phoenix, all the core stuff is really high quality, but in many cases you might doing more work that you could have just pulled from a gem in Ruby. It's unfortunate IMO. It's an underrated language.
  - pythonaut_16 2y ago
    Very much this. I think the community tends to overestimate the ecosystem’s maturity which is one of the big things holding it back, both because it blinds the community to areas that need improvement and leads to bigger shocks when newcomers do unexpectedly run into the rough edges.
  - arrowsmith 2y ago
    This is true, but AI is closing the gap. "Take this Ruby gem and write the equivalent functionality in Elixir" is something that LLMs are very good at.
    - weatherlite 2y ago
      That's over simplifying things ... no, complex gems cannot simply be transformed to Elixir with ChatGPT. You'd have to have an expert in both languages fixing all the bugs.
  - cess11 2y ago
    Do you have some examples?
paradox460 2y ago
And the switch is rather easy. I've been writing elixir for nearly 10 years, rails before that, and have overseen the "conversion" of several engineers from one to the other. Generally I'd say any senior rails dev, given the right task, can write decent elixir code on their first day. There are a lot fewer foot guns in elixir and Phoenix, and so other than language ergonomics (a simple rule that doesn't stretch far but works at the beginning is use pipe instead of dot), there's minimal barriers
- troad 2y ago
  Honest question from someone working on a non-negligible Rails codebase: what would be my gains, were I to switch to Elixir? I've watched Elixir with much interest from afar, I even recently cracked open the book on it, but I feel like my biggest pain points with Ruby are performance and lack of gradual typing (and consequent lack of static analysis, painful refactoring, etc), and it doesn't really seem like Elixir has much to offer on those. What does Elixir solve, that Ruby struggles on?
  - paradox460 2y ago
    Performance is a complex story. Elixir is very good at massive amounts of parallel computing, and uses this to handle each request (or socket) in it's own contained manner, which nets you some simplicity in designing systems and scaling. However it's not very good at single threaded off-the-block performance (but neither is Ruby) Typing is coming, some is already here, and if you're impatient you can use dialyzer to get half way there. But in my experience you need it less than you'd think. Elixir primitives are rather expressive, and due to the functional nature there really isn't any "new" data structure that isn't easy to dive into. And with each function head being easy to pattern match in, you can dictate the shape of data flow through your application much as you'd do in a typed language. The ide story isn't great, but it's getting better. There are a few LSPs out there, and an official one coming soon. And I'd say all of them beat solargraph handily But most of all I'd say that, since it's a bit more strict, elixir saves you and your coworkers from yourselves and each other. There are still several ways to do something, like Ruby, but going out of your way to write very cutesy code that the next programmer will loathe is more difficult. Not impossible, but harder. And with the rock stable concurrency systems in place, a lot of the impetus to come up with clever solutions isn't there

jackbravo 2y ago

If you want something more similar to Next.JS but in the python world, now you have https://fastht.ml/ https://fastht.ml/, which also has a big performance benefit over Django. Hahaha, same as Next.JS over Rails, because it is much more bare bones. But I would say that fasthtml has the advantage of being super easy to integrate more AI libraries from the python world.

zzzeek 2y ago
now that was a crazy rabbit hole

simonw 2y ago

Django shouldn't even require gevent - Django's ASGI support has been baking for a few releases now and supports async views which should be well suited to proxying streams from LLMs etc. Relevant: - https://fly.io/django-beats/running-tasks-concurrently-in-django-asynchronous-views/ https://fly.io/django-beats/running-tasks-concurrently-in-dj... - https://blog.pecar.me/django-streaming-responses https://blog.pecar.me/django-streaming-responses (Reminds me I should try that out properly myself.)

pphysch 2y ago
then you have to rewrite your whole app to use asyncio keywords and colored ORM methods. A gevent monkey patch, or eventually nogil concurrency makes a lot more practical sense.
- simonw 2y ago
  You don't have to rewrite your whole app - you can continue using the regular stuff in synchronous view functions, then have a few small async views for your LLM streaming pieces. I've never quite gotten comfortable with gevent patches, but that's more because I don't personally understand them or what their edge cases might be than a commentary on their reliability.
  - pphysch 2y ago
    Last time I checked, simply running Django through the ASGI interface dealt you a baseline perf hit.
    - simonw 2y ago
      It's pretty minimal: https://docs.djangoproject.com/en/5.1/topics/async/#performance https://docs.djangoproject.com/en/5.1/topics/async/#performa... > When running in a mode that does not match the view (e.g. an async view under WSGI, or a traditional sync view under ASGI), Django must emulate the other call style to allow your code to run. This context-switch causes a small performance penalty of around a millisecond.

troad 2y ago

> You have the Python type system, and while it's inferior to TypeScript's in many ways, it's far more ubiquitous than Ruby's Sorbet. I'm a big fan of Ruby, but God I wish it had good, in-line type hinting. Sorbet annotations are too noisy and the whole thing feels very awkwardly bolted on, while RBS' use of external files make it a non-starter.

lloeki 2y ago
I for one really like RBS being external files, it keeps the Ruby side of things uncluttered. When I do need types inline I believe it is the editor's job to show them dynamically, e.g via facilities like tooltips, autocompletion, or vim inlay hints and virtual text, which can apply to much more than just signatures near method definitions. Types are much more useful where the code is used than where it is defined. I follow a 1:1 lib/.rb - sig/.rbs convention and have projection+ files to jump from one to the other instantly. And since the syntax of RBS is so close to Ruby I found myself accidentally writing things type-first then using that as a template to write the actual code. Of note, if you crawl soutaro's repo (author of steep) you'll find a prototype of inline RBS. + used by vim projectionist and vscode projection extension
sankha93 2y ago
Do you mean Ruby lacks syntactic support for adding type annotations inline in your programs? I am one of the authors of RDL (https://github.com/tupl-tufts/rdl https://github.com/tupl-tufts/rdl) a research project that looked at type systems for Ruby before it became mainstream. We went for strings that looked nice, but were parsed into a type signature. Sorbet, on the other hand, uses Ruby values in a DSL to define types. We were of the impression that many of our core ideas were absorbed by other projects and Sorbet and RBS has pretty much mainstream. What is missing to get usable gradual types in Ruby?
- pmontra 2y ago
  None of my customers ever asked for type definitions in Ruby (nor in Python.) I'm pretty happy of the choice of hiding types under the carpet of a separate file. I think they made it deliberately because Ruby's core team didn't like type definitions but had to cave to the recent fashion. It will swing back but I think that this is a slow pendulum. Talking about myself I picked Ruby 20 years ago exactly because I didn't have to type types so I'm not a fan of the projects you are working at, but I don't even oppose them. I just wish I'm never forced to define types.