5 ms·

I love how pragmatic the approach is here. We still don't have parser generators which generate human understandable/readable recursive descent parser with good

I love how pragmatic the approach is here. We still don't have parser generators which generate human understandable/readable recursive descent parser with good error recovery baked in. But i am guessing the ruby syntax is too complicated/irregular anyway. On a macro side, i am always surprised with the length and effort company will go to keep working with solution design and selected at their young age, and try to scale them well beyond the braking point. And i feel like most of this effort to keep ruby alive could have been used to slowly transition part of the infra to something JVM/.Net/Rust based for much more bang for engineering buck.

ecshafer 3y ago
(Disclaimer: Shopify employee but not part of this team) Ruby is my favorite language so far to write "business logic" in, and Rails is a very developer friendly web framework that makes it very easy to set up websites. So I think this is a pretty great strategy overall. The end of the day I think the way to get tech too scale, is for companies / academics / etc to really just put the work in. Java was very slow on release, but now the JVM is blazingly fast, and that is because of the huge amount of work put in by Sun, Oracle, IBM, Redhat, Google, Azul, etc. over the years. For what its worth though, we do also have Go, Rust, Scala, etc. in different places in our stack where its needed.
- soulbadguy 3y ago
  Disclaimer : my career has mainly been on compiler/runtime and other infrastructure related stuff, so not much of a business logic/web dev kind of guy. > The end of the day I think the way to get tech too scale, is for companies / academics / etc to really just put the work in. This is true in the abstract, but my point is mainly about the comparative effectiveness of different strategies when it come to building scalable infrastructure (or scaling up infra): Keep the bulk of the existing code and invest in making the underlying compiler/interpreter/tool chain better or progressively migrate the code to a tool chain with better scaling capabilities from the get go. From experience, the nature and semantic of a language severely what is "reasonably" possible in the runtime in term of safety (like code loader in java), performance (both CPU and memory) and tooling. Now improvement are possible, but they tend become exponentially expensive as time goes on. Now, obviously there is always a tension between developer productivity and infra concerns, but i do believe that we have better compromise point on that line with newer language and framework. I totally agree that in the 2000's the experience of most "enterprise framework" sucked very hard, and the emergence of language such ruby/python etc... was a god send : a response to the overly rigid and ceremonial way of the past. But with time, we have been able to understand better what makes a good programming experience, distill that into better designed languages and framework which offer "better" compromises. For example : - Instead of dynamic vs static type , we have progressively typed and type inference - Instead of GC vs non-GC we have rust borrow-checker - Instead of runtime meta programing with have DSL and macro's Even more important, i believe that a lot of the experience come from the tooling around the language, and there again the "cargo/dotnet/go" cli approach with a single coherent entry point for both package management and framework scaffolding ease a lot of the pain of the old way. With all of that we now have languages which offer a better compromise on the dev. prod vs infra/performance... > JVM is blazingly fast I would say blazingly faster... But compared to C++ (or even rust) java is still quite slow. Especially for anything compute intensive.
  - byroot 3y ago
    > Disclaimer : my career has mainly been on compiler/runtime and other infrastructure related stuff, so not much of a business logic/web dev kind of guy. That probably explain your surprise. This question comes up extremely often on HN, the answer is that even if you were to stop all feature development, re-platforming a large organization like Shopify would take years, not even considering all the re-training, lost knowledge etc. And you'd spend these years having to support multiple platforms so that would only pay off much later, and in the meantime your competitors continue iterating. In general it's best to stick with the devil you know, unless the new platform interoperate very well with the old one.
    - soulbadguy 3y ago
      > re-platforming a large organization like Shopify would take years I am not sure what re-platforming means here, but "slowly transitioning" to a different stack is not the same as stop and rewrite everything. > That probably explain your surprise While i don't write "business logic", i have been involved in a lot of project to babysit, maintain, refactor and/or improve business code bases. Sometime the runtime can only do so much and you have to adjust the user level code. And from experience it's never as bad as some make it to be. > And you'd spend these years having to support multiple platforms so that would only pay off much later, and in the meantime your competitors continue iterating. Again i see this very often as well. Software engineering seem to over emphasizes the cost of platform transition while downplaying the operating burden of not modernizing.
      - byroot 3y ago
        > over emphasizes the cost of platform transition while downplaying the operating burden of not modernizing. Given that I work on the Ruby Infrastructure team at Shopify, I think I have a well informed view on both the cost of transitioning and the operating burden.
    - RhodesianHunter 3y ago
      When you reach the level of scale where you need to move to distributed systems, you pretty much need to re-platform anyways. That's usually the ideal time to make a decision like this.
aantix 3y ago
Where is the “more bang for the engineering buck”?
- mperham 3y ago
  JRuby. TruffleRuby.
- soulbadguy 3y ago
  ?
pulisse 3y ago
The number of engineer years required for the parser rewrite was probably in the low double digits. Migrating a platform like Spotify to a new framework in a different language would take hundreds.

ecshafer 3y ago

(Disclaimer: Shopify employee but not part of this team) Ruby is my favorite language so far to write "business logic" in, and Rails is a very developer friendly web framework that makes it very easy to set up websites. So I think this is a pretty great strategy overall. The end of the day I think the way to get tech too scale, is for companies / academics / etc to really just put the work in. Java was very slow on release, but now the JVM is blazingly fast, and that is because of the huge amount of work put in by Sun, Oracle, IBM, Redhat, Google, Azul, etc. over the years. For what its worth though, we do also have Go, Rust, Scala, etc. in different places in our stack where its needed.

soulbadguy 3y ago
Disclaimer : my career has mainly been on compiler/runtime and other infrastructure related stuff, so not much of a business logic/web dev kind of guy. > The end of the day I think the way to get tech too scale, is for companies / academics / etc to really just put the work in. This is true in the abstract, but my point is mainly about the comparative effectiveness of different strategies when it come to building scalable infrastructure (or scaling up infra): Keep the bulk of the existing code and invest in making the underlying compiler/interpreter/tool chain better or progressively migrate the code to a tool chain with better scaling capabilities from the get go. From experience, the nature and semantic of a language severely what is "reasonably" possible in the runtime in term of safety (like code loader in java), performance (both CPU and memory) and tooling. Now improvement are possible, but they tend become exponentially expensive as time goes on. Now, obviously there is always a tension between developer productivity and infra concerns, but i do believe that we have better compromise point on that line with newer language and framework. I totally agree that in the 2000's the experience of most "enterprise framework" sucked very hard, and the emergence of language such ruby/python etc... was a god send : a response to the overly rigid and ceremonial way of the past. But with time, we have been able to understand better what makes a good programming experience, distill that into better designed languages and framework which offer "better" compromises. For example : - Instead of dynamic vs static type , we have progressively typed and type inference - Instead of GC vs non-GC we have rust borrow-checker - Instead of runtime meta programing with have DSL and macro's Even more important, i believe that a lot of the experience come from the tooling around the language, and there again the "cargo/dotnet/go" cli approach with a single coherent entry point for both package management and framework scaffolding ease a lot of the pain of the old way. With all of that we now have languages which offer a better compromise on the dev. prod vs infra/performance... > JVM is blazingly fast I would say blazingly faster... But compared to C++ (or even rust) java is still quite slow. Especially for anything compute intensive.
- byroot 3y ago
  > Disclaimer : my career has mainly been on compiler/runtime and other infrastructure related stuff, so not much of a business logic/web dev kind of guy. That probably explain your surprise. This question comes up extremely often on HN, the answer is that even if you were to stop all feature development, re-platforming a large organization like Shopify would take years, not even considering all the re-training, lost knowledge etc. And you'd spend these years having to support multiple platforms so that would only pay off much later, and in the meantime your competitors continue iterating. In general it's best to stick with the devil you know, unless the new platform interoperate very well with the old one.
  - soulbadguy 3y ago
    > re-platforming a large organization like Shopify would take years I am not sure what re-platforming means here, but "slowly transitioning" to a different stack is not the same as stop and rewrite everything. > That probably explain your surprise While i don't write "business logic", i have been involved in a lot of project to babysit, maintain, refactor and/or improve business code bases. Sometime the runtime can only do so much and you have to adjust the user level code. And from experience it's never as bad as some make it to be. > And you'd spend these years having to support multiple platforms so that would only pay off much later, and in the meantime your competitors continue iterating. Again i see this very often as well. Software engineering seem to over emphasizes the cost of platform transition while downplaying the operating burden of not modernizing.
    - byroot 3y ago
      > over emphasizes the cost of platform transition while downplaying the operating burden of not modernizing. Given that I work on the Ruby Infrastructure team at Shopify, I think I have a well informed view on both the cost of transitioning and the operating burden.
  - RhodesianHunter 3y ago
    When you reach the level of scale where you need to move to distributed systems, you pretty much need to re-platform anyways. That's usually the ideal time to make a decision like this.

aantix 3y ago

Where is the “more bang for the engineering buck”?

mperham 3y ago
JRuby. TruffleRuby.
soulbadguy 3y ago
?

pulisse 3y ago

The number of engineer years required for the parser rewrite was probably in the low double digits. Migrating a platform like Spotify to a new framework in a different language would take hundreds.