7 ms·

How often does "hit the wall from a performance and maintainability POV" happen with Python / Ruby / NodeJS ? I mean seriously, how many more huge projects do w

How often does "hit the wall from a performance and maintainability POV" happen with Python / Ruby / NodeJS ? I mean seriously, how many more huge projects do we need to build until we can show it's a sane alternative? 120 Billion worth Shopify isn't enough, maybe Instagram then? Github ?

cmrdporcupine 6y ago
It entirely depends on what you're doing. There are two issues as I see it: a) Lack of static analysis tools means having to do more testing (manual or automated) for simple mechanical errors. b) For super high throughput low latency systems, they are not up to snuff. When I worked in real time bidding in ad-tech this was actually a serious concern. I'd reach for Rust (or C/C++) in this scenario now.
- joelbluminator 6y ago
  Yes there are niches where Ruby won't work, even big niches like Kernel development or real time systems maybe. But for web development, in general, these languages have proven themselves for so long it's getting quite ridiculous now to say they won't work. As for testing - I disagree. Frameworks like Rails are so easily testable it's a breeze. Java/Spring dependency injection is jumping through hoops just to provide a testable framework I find it hard to believe it's any easier.
  - jimbokun 6y ago
    > But for web development, in general, these languages have proven themselves for so long it's getting quite ridiculous now to say they won't work. And I don't think web development is where Go is getting traction. More in back end services and infrastructure. > Java/Spring dependency injection is jumping through hoops just to provide a testable framework I find it hard to believe it's any easier. As someone who develops with Spring everyday, I don't think it's the best solution for anything.
    - joelbluminator 6y ago
      What are your main pain points with Spring?
      - jimbokun 6y ago
        Annotation oriented programming. Having the code execution paths radically change by adding an annotation to a method or a class makes it very difficult to reason about what it will do when deployed. If that annotation you found through Google does what you want and expect, that's great. But if it doesn't, or fails in unexpected ways, debugging it can be a nightmare.
        joelbluminator 6y ago
        Its interesting that a java framework is so "magical", feels like it goes against the java mindset I know.
        jimbokun 6y ago
        Yes, that's what makes it so terrible. All of the "action at a distance" complexity of Ruby meta-programming, with none of the concise and easy to read code!
        cmrdporcupine 6y ago
        This always baffled me -- when Guava came on the scene, and when Spring also adopted annotations... The whole original point of dependency injection was to decouple dependency management from the code, to make it easier to test, and easier to reason about and analyze. DI via annotations goes ahead and sticks them right back in there. And now we have, like you say, magic code that is difficult to reasona bout. XML sucked, but annotations sucked more.
  - cmrdporcupine 6y ago
    Yes, I don't do web development. At least not mostly. But I understand that that's what most people out there are doing. But they should also not delude theirselves that the toolsets appropriate for that environment are appropriate every where else. I see this bias a lot, on HN even, that everyone now is a "full stack developer" doing this kind of development. Many of us work in entirely different stacks.
- matthewaveryusa 6y ago
  just out of curiosity -- for ad-tech real time bidding, what's your network latency and bandwidth like? I can buy that C/C++ is needed if you are colocated to an exchange, but if you're bidding online, those few fractions of an ms you save in C/C++ vs node.js you could have also saved by locating your server closer to whatever ad-tech exchange you're bidding on.
  - synthc 6y ago
    when I worked in adtech, we located the servers close to the exchange. This indeed made much more difference to the latency than code optimizations.
  - cmrdporcupine 6y ago
    The issue I had back in this line of work was with garbage collection and was an issue in the 99th percentile. When you have an expectation from the exchange that you respond in under 80ms, and 25-50ms of that is eaten up by transport, you don't have a lot of time to mess around. So you spend the first chunk of time optimizing I/O and how you're accessing it. Then you start looking at computation -- improving caching, etc. At a certain point you start noticing in your graphs that you're on average doing quite well. But there's those hiccups every Nth request... And now you're in the game of fighting with your language's garbage collection algorithm...
    - politician 6y ago
      At this level of scrutiny, replacing algorithms is appropriate and if the algorithms are built into the language then replacing the language may be one* way to clear the issue, but you have to do your homework to get there. *sync.Pool, arena allocation strategies
    - user5994461 6y ago
      One clean and dirty way to help with that. Try to improve allocations where possible. There's often plenty of small allocations happening in Java that can be avoided (string usage is one major driver). Less allocations means less frequent garbage collections. Then one hack is to disable the garbage collection. Let the software run up to consume all the 100 GB of memory of the server and crash and restart. There is no pause of garbage collection when there is no garbage collection. If it's not enough, the last resort is to write native code or switch to C++.
- synthc 6y ago
  Speaking from experience, the JVM is fast enough for real time bidding in adtech. Most major ad exchanges require the latency to be below ~80ms, which is not that hard to achieve. In contrast to high-frequency trading, there is no competive advantage in having a lower latency.
  - kelvie 6y ago
    Are compile times and program start-up times not a factor? One of the things I really appreciate about golang (from a completely different field) is how quic teh builds are, and how fast binaries start up (it's like I wrote it in C).
    - user5994461 6y ago
      Java can compile quickly, a few minutes at most when C++ would take hours, so I am tempted to say that's not a problem. The startup time is negligible in my experience (few seconds for JVM or python imports). I have to take over slow starting applications from time to time and it's always because of loading data and doing stupid shit on startup, irrelevant of the language. It's not a problem for production because server application only reboot once in forever.
      - tsimionescu 6y ago
        It's still a problem for microservices architectures, unfortunately, especially if you want to support dynamic scaling of some kind. A few seconds is nothing if you expect that your server will be up forever, but it becomes a lot if sometimes it goes down for a bit to move to a different machine, and that takes seconds for your customers. Also, JIT languages have a very poor habit of doing a terrible first impression because of the warm-up time, especially in Java. If you are delivering applications to customers, that becomes a real burden - the very first time they use your shiny new application, everything is moving like molasses, until the JVM decides it's JIT time...
        thebean11 6y ago
- throw_m239339 6y ago
  > b) For super high throughput low latency systems, they are not up to snuff. When I worked in real time bidding in ad-tech this was actually a serious concern. I'd reach for Rust (or C/C++) in this scenario now. What did you do? You wrote servers in C? you wrote a Redis for instance?
  - cmrdporcupine 6y ago
    Optimized for GC and figured it out. But that point also coincided with a job switch; I went to another ad tech company, but on the exchange side rather than the bidding side. And all our ad server infrastructure for that was written in C/C++ (with embeded V8 JS for biz logic stuff). Then that company was bought by Google, and I worked on the exchange side at Google, too, where everything was also in C++. My successor at the original startup rewrote everything in Python. And I watched from the exchange side as they struggled for two months to meet basic performance constraints. They eventually got it though. It certainly can be done. It's worth pointing out this was 10 years ago. And in the meantime we've had the usual improvements in machine performance, and SSDs are a thing in data centres, etc.
Thaxll 6y ago
When you see the effort some of those compagnies go to get some decent performance, chose the right language from the start would have been easier on that side, look at Facebook with Hack, Youtube ( Python ) that re-implemented a VM in Go, Shopify is also doing a lot of things to improve performance because of Ruby limitation, when you pick up a language with by default good performance you don't have to go that route.
- claudiug 6y ago
  but ruby is moving in this direction - jit - more functional tools - better introspection - sorbet - ractor - truffleruby, and jruby
  - ksec 6y ago
    Unfortunately, in the case of Shopify none of those tools will run rails any faster. I think they only use TruffleRuby for some internal tools. Hopefully things will improve now ChrisSeaton is working there.
  - joelbluminator 6y ago
    Ruby is working hard on performance but Sorbet is hardly a common thing. The Rails community isn't gonna adopt types anytime soon imo. To be fair most likely never, at least not as a community. Some companies may go that route but I hardly anticipate a big movement.
  - satvikpendem 6y ago
    Those are all grafted on afterwards, and they don't see too much use apart from the people who made them (ie Stripe with sorbet). Compare that to a language that has first-class support for all of these built in. As you learn such language you're forced to learn these concepts as well, so more people in the community use them, thereby increasing their effectiveness through sustained development because they're popular features.
- joelbluminator 6y ago
  Sure, all web development should be C with some assembly sprinkles.
jimbokun 6y ago
> 120 Billion worth Shopify isn't enough, maybe Instagram then? Github ? Do these sites also have back end services written in other languages for the heavy listing? Anecdotally, seems to be a lot of stories to that effect. And as noted in the article, Javascript/NodeJS is in a different performance category from Python and Java, more like Java or Go. But the simplicity of Go might make the performance and memory usage more predictable.
- joelbluminator 6y ago
  Shopify is absolutely a Ruby / Rails company. If out of the thousands of developers that work there you have a team or two writing some performance heavy c++ / Go code that says absolutely nothing.
  - politician 6y ago
    Wouldn't it actually say a lot? Engineering is about trade-offs, and the trade-off that says "use this special language for 1% of our most critical services instead of the company-wide standard language" is not "absolutely nothing". Lines of Code is a bad proxy for mission critical.
    - joelbluminator 6y ago
      All of Shopify is mission critical. The code that constitutes the Shopify monolith is mission critical for sure. The company won't function without it. How do you define mission critical?
      - politician 6y ago
        This is somewhat circular, but: when your team decides to rewrite a component in a more complicated or more obscure (at least at Shopify) language to increase throughput rather than spinning up more instances of the current solution, then you're talking about a component that's even more critical than the other critical components.

cmrdporcupine 6y ago

It entirely depends on what you're doing. There are two issues as I see it: a) Lack of static analysis tools means having to do more testing (manual or automated) for simple mechanical errors. b) For super high throughput low latency systems, they are not up to snuff. When I worked in real time bidding in ad-tech this was actually a serious concern. I'd reach for Rust (or C/C++) in this scenario now.

joelbluminator 6y ago
Yes there are niches where Ruby won't work, even big niches like Kernel development or real time systems maybe. But for web development, in general, these languages have proven themselves for so long it's getting quite ridiculous now to say they won't work. As for testing - I disagree. Frameworks like Rails are so easily testable it's a breeze. Java/Spring dependency injection is jumping through hoops just to provide a testable framework I find it hard to believe it's any easier.
- jimbokun 6y ago
  > But for web development, in general, these languages have proven themselves for so long it's getting quite ridiculous now to say they won't work. And I don't think web development is where Go is getting traction. More in back end services and infrastructure. > Java/Spring dependency injection is jumping through hoops just to provide a testable framework I find it hard to believe it's any easier. As someone who develops with Spring everyday, I don't think it's the best solution for anything.
  - joelbluminator 6y ago
    What are your main pain points with Spring?
    - jimbokun 6y ago
      Annotation oriented programming. Having the code execution paths radically change by adding an annotation to a method or a class makes it very difficult to reason about what it will do when deployed. If that annotation you found through Google does what you want and expect, that's great. But if it doesn't, or fails in unexpected ways, debugging it can be a nightmare.
      - joelbluminator 6y ago
        Its interesting that a java framework is so "magical", feels like it goes against the java mindset I know.
        jimbokun 6y ago
        Yes, that's what makes it so terrible. All of the "action at a distance" complexity of Ruby meta-programming, with none of the concise and easy to read code!
      - cmrdporcupine 6y ago
        This always baffled me -- when Guava came on the scene, and when Spring also adopted annotations... The whole original point of dependency injection was to decouple dependency management from the code, to make it easier to test, and easier to reason about and analyze. DI via annotations goes ahead and sticks them right back in there. And now we have, like you say, magic code that is difficult to reasona bout. XML sucked, but annotations sucked more.
- cmrdporcupine 6y ago
  Yes, I don't do web development. At least not mostly. But I understand that that's what most people out there are doing. But they should also not delude theirselves that the toolsets appropriate for that environment are appropriate every where else. I see this bias a lot, on HN even, that everyone now is a "full stack developer" doing this kind of development. Many of us work in entirely different stacks.
matthewaveryusa 6y ago
just out of curiosity -- for ad-tech real time bidding, what's your network latency and bandwidth like? I can buy that C/C++ is needed if you are colocated to an exchange, but if you're bidding online, those few fractions of an ms you save in C/C++ vs node.js you could have also saved by locating your server closer to whatever ad-tech exchange you're bidding on.
- synthc 6y ago
  when I worked in adtech, we located the servers close to the exchange. This indeed made much more difference to the latency than code optimizations.
- cmrdporcupine 6y ago
  The issue I had back in this line of work was with garbage collection and was an issue in the 99th percentile. When you have an expectation from the exchange that you respond in under 80ms, and 25-50ms of that is eaten up by transport, you don't have a lot of time to mess around. So you spend the first chunk of time optimizing I/O and how you're accessing it. Then you start looking at computation -- improving caching, etc. At a certain point you start noticing in your graphs that you're on average doing quite well. But there's those hiccups every Nth request... And now you're in the game of fighting with your language's garbage collection algorithm...
  - politician 6y ago
    At this level of scrutiny, replacing algorithms is appropriate and if the algorithms are built into the language then replacing the language may be one* way to clear the issue, but you have to do your homework to get there. *sync.Pool, arena allocation strategies
  - user5994461 6y ago
    One clean and dirty way to help with that. Try to improve allocations where possible. There's often plenty of small allocations happening in Java that can be avoided (string usage is one major driver). Less allocations means less frequent garbage collections. Then one hack is to disable the garbage collection. Let the software run up to consume all the 100 GB of memory of the server and crash and restart. There is no pause of garbage collection when there is no garbage collection. If it's not enough, the last resort is to write native code or switch to C++.
synthc 6y ago
Speaking from experience, the JVM is fast enough for real time bidding in adtech. Most major ad exchanges require the latency to be below ~80ms, which is not that hard to achieve. In contrast to high-frequency trading, there is no competive advantage in having a lower latency.
- kelvie 6y ago
  Are compile times and program start-up times not a factor? One of the things I really appreciate about golang (from a completely different field) is how quic teh builds are, and how fast binaries start up (it's like I wrote it in C).
  - user5994461 6y ago
    Java can compile quickly, a few minutes at most when C++ would take hours, so I am tempted to say that's not a problem. The startup time is negligible in my experience (few seconds for JVM or python imports). I have to take over slow starting applications from time to time and it's always because of loading data and doing stupid shit on startup, irrelevant of the language. It's not a problem for production because server application only reboot once in forever.
    - tsimionescu 6y ago
      It's still a problem for microservices architectures, unfortunately, especially if you want to support dynamic scaling of some kind. A few seconds is nothing if you expect that your server will be up forever, but it becomes a lot if sometimes it goes down for a bit to move to a different machine, and that takes seconds for your customers. Also, JIT languages have a very poor habit of doing a terrible first impression because of the warm-up time, especially in Java. If you are delivering applications to customers, that becomes a real burden - the very first time they use your shiny new application, everything is moving like molasses, until the JVM decides it's JIT time...
      - thebean11 6y ago
throw_m239339 6y ago
> b) For super high throughput low latency systems, they are not up to snuff. When I worked in real time bidding in ad-tech this was actually a serious concern. I'd reach for Rust (or C/C++) in this scenario now. What did you do? You wrote servers in C? you wrote a Redis for instance?
- cmrdporcupine 6y ago
  Optimized for GC and figured it out. But that point also coincided with a job switch; I went to another ad tech company, but on the exchange side rather than the bidding side. And all our ad server infrastructure for that was written in C/C++ (with embeded V8 JS for biz logic stuff). Then that company was bought by Google, and I worked on the exchange side at Google, too, where everything was also in C++. My successor at the original startup rewrote everything in Python. And I watched from the exchange side as they struggled for two months to meet basic performance constraints. They eventually got it though. It certainly can be done. It's worth pointing out this was 10 years ago. And in the meantime we've had the usual improvements in machine performance, and SSDs are a thing in data centres, etc.

Thaxll 6y ago

When you see the effort some of those compagnies go to get some decent performance, chose the right language from the start would have been easier on that side, look at Facebook with Hack, Youtube ( Python ) that re-implemented a VM in Go, Shopify is also doing a lot of things to improve performance because of Ruby limitation, when you pick up a language with by default good performance you don't have to go that route.

claudiug 6y ago
but ruby is moving in this direction - jit - more functional tools - better introspection - sorbet - ractor - truffleruby, and jruby
- ksec 6y ago
  Unfortunately, in the case of Shopify none of those tools will run rails any faster. I think they only use TruffleRuby for some internal tools. Hopefully things will improve now ChrisSeaton is working there.
- joelbluminator 6y ago
  Ruby is working hard on performance but Sorbet is hardly a common thing. The Rails community isn't gonna adopt types anytime soon imo. To be fair most likely never, at least not as a community. Some companies may go that route but I hardly anticipate a big movement.
- satvikpendem 6y ago
  Those are all grafted on afterwards, and they don't see too much use apart from the people who made them (ie Stripe with sorbet). Compare that to a language that has first-class support for all of these built in. As you learn such language you're forced to learn these concepts as well, so more people in the community use them, thereby increasing their effectiveness through sustained development because they're popular features.
joelbluminator 6y ago
Sure, all web development should be C with some assembly sprinkles.

jimbokun 6y ago

> 120 Billion worth Shopify isn't enough, maybe Instagram then? Github ? Do these sites also have back end services written in other languages for the heavy listing? Anecdotally, seems to be a lot of stories to that effect. And as noted in the article, Javascript/NodeJS is in a different performance category from Python and Java, more like Java or Go. But the simplicity of Go might make the performance and memory usage more predictable.

joelbluminator 6y ago
Shopify is absolutely a Ruby / Rails company. If out of the thousands of developers that work there you have a team or two writing some performance heavy c++ / Go code that says absolutely nothing.
- politician 6y ago
  Wouldn't it actually say a lot? Engineering is about trade-offs, and the trade-off that says "use this special language for 1% of our most critical services instead of the company-wide standard language" is not "absolutely nothing". Lines of Code is a bad proxy for mission critical.
  - joelbluminator 6y ago
    All of Shopify is mission critical. The code that constitutes the Shopify monolith is mission critical for sure. The company won't function without it. How do you define mission critical?
    - politician 6y ago
      This is somewhat circular, but: when your team decides to rewrite a component in a more complicated or more obscure (at least at Shopify) language to increase throughput rather than spinning up more instances of the current solution, then you're talking about a component that's even more critical than the other critical components.