8 ms·

> In comparison with the previous Java service, the updated backend delivers a 40% increase in performance, along with improved scalability, security, and avail

• tiffanyh 1y ago
  I'd typically agree with your comment but ... Given that they also experienced a 90% reduction in Memory Usage (presumably from Java GC vs Swift AOT memory management) - it seems more likely the gains are in fact from the difference in languages.
  • plorkyeran 1y ago
    The typical rule of thumb is that getting good performance out of tracing GC requires doubling your memory usage, so a 90% reduction suggests that they made significant improvements on top of the language switch.
    • username223 1y ago
      I remember the 2x rule from 20 years ago - do you know if things have changed? If locality is more important now, tracing GC might never be as performant as reference counting. Either you use 2x the memory and thrash your cache, or you use less and spend too much CPU time collecting.
      • jeroenhd 1y ago
        Java has had AOT compilation for a while, so traditional GC and its massive overhead are no longer a strict necessity. Even AOT Java will probably stay behind Swift or any other natively compiled language in terms of memory usage, but it shouldn't be that drastic. As for performance and locality, Java's on-the-fly pointer reordering/compression can give it an edge over even some compiled languages in certain algorithms. Hard to say if that's relevant for whatever web framework Apple based their service on, but I wouldn't discount Java's locality optimisations just because it uses a GC.
        • username223 1y ago
          > Java has had AOT compilation for a while, so traditional GC and its massive overhead are no longer a strict necessity. You mean it does escape analysis and stack-allocates what it can? That would definitely help, but not eliminate the GC. Or are you thinking of something else? Thinking about it more, I remember that Java also has some performance-hostile design decisions baked in (e.g. almost everything's an Object, arrays aren't packed, dynamic dispatch everywhere). Swift doesn't have that legacy to deal with.
          • acdha 1y ago
            Java also has a lot of culture making optimization-resistant code so there’s the question of whether you’re talking about the language itself or various widespread libraries, especially if they’re old enough to have patterns designed around aesthetics or now-moot language limitations rather than performance. I’ve replaced Java code with Python a few times and each time even though we did it for maintenance (more Python devs available) we saw memory usage more than halved while performance at least doubled because the code used simpler functions and structures. Java has a far more advanced GC and JIT but at some point the weight of code and indirection wins out.
            • winrid 1y ago
              That's interesting. I did a line by line rewrite of a large Django route to Quarkus and it was 10x faster, not using async or anything.
              • acdha 1y ago
                That’s why I said “culture” - by all rights the JVM should win that competition. I wrote a bit more about the most recent one in a sibling comment but I’d summarize it as “the JVM can’t stop an enterprise Java developer”. https://news.ycombinator.com/item?id=44179589 https://news.ycombinator.com/item?id=44179589
                • 1y ago
      • cogman10 1y ago
        It all depends, but one major advantage of the way the JVM GCs is related memory will tend to be colocated. This is particularly true of the serial, parallel, and G1GC collectors. Let's say you have an object that looks like A -> B -> C. Even if the allocation of A/B/C happened at very temporally different times and inbetween different allocations, the next time the GC runs as it traverses the graph it will see and place in memory [A, B, C] assuming A is still live. That means even if the memory originally looks something like [A, D, B, Q, R, S, T, C] the act of collecting and compacting has a tendency to colocate.
        • username223 1y ago
          That's the theory -- a compacting collector will reduce fragmentation and can put linked objects next to each other. On the other hand, a reference count lives in the object, so you're likely using that cache line already when you change it. I don't know which of these is more important on a modern machine, and it probably depends upon the workload.
          • cogman10 1y ago
            The problem is memory colocation not RC management. But I agree, it'll likely be workload dependent. One major positive aspect of RC is the execution costs are very predictable. There's little external state which can negatively impact performance (like the GC currently running). The downside is fragmentation and the CPU time required for memory management. If you have an A -> B -> C chain where A is the only owner of the B and B is the only owner of C, then when A hits 0, it has to do 2 pointer hops to deallocate B and then deallocate C (plus arena management for the deallocs). One of the big benefits of JVM moving style collectors is that when A dies, the collector does not need to visit B or C to deallocate them. The collector only visits and moves live memory.
            • dwaite 1y ago
              > The downside is fragmentation and the CPU time required for memory management. If you have an A -> B -> C chain where A is the only owner of the B and B is the only owner of C, then when A hits 0, it has to do 2 pointer hops to deallocate B and then deallocate C (plus arena management for the deallocs). I suspect this puts greater emphasis on functionality like value types and flexibility in compositionally creating objects. You can trend toward larger objects rather than nesting inner objects for functionality. For example, you can use tagged unions to represent optionality rather than pointers. The cost of deep A->B->C relationships in Java comes during collections, which still default to be halting. The difference is a reference counting GC will evaluate these chains while removing objects, while a reference tracking GC will evaluate live objects. So, garbage collection is expensive for ref-counting if you are creating large transient datasets, and is expensive for ref-tracking GC if you are retaining large datasets.
    • geodel 1y ago
      That is just GC part. Another big difference is Reference type (Java) vs Value Type (Swift).
    • mrighele 1y ago
      The 90% reduction doesn't necessarily have to be related only to GC. In my experience Java is a memory hog even compared to other garbage collected languages (that's my main gripe about the language). I think a good part of the reason is that if you exclude primitive types, almost everything in Java is an heap-allocated object and Java objects are fairly "fat": every single instance has an header of between 96 and 128 bit on 64-bit architectures [1]. That's... a lot. Just by making the headers smaller (the topic of the above link) you can get 20% decrease in heap usage and improvements in cpu and GC time [2]. My hope is that once value classes arrive [3][4], and libraries start to use them, we will see a substantial decrease in heap usage in an average java app. [1] https://openjdk.org/jeps/450 https://openjdk.org/jeps/450 [2] https://openjdk.org/jeps/519 https://openjdk.org/jeps/519 [3] https://openjdk.org/jeps/401 https://openjdk.org/jeps/401 [4] https://www.youtube.com/watch?v=Dhn-JgZaBWo https://www.youtube.com/watch?v=Dhn-JgZaBWo
      • formerly_proven 1y ago
        If Java's 128-bit object headers are already fairly fat, then what adjective applies to CPython's? [] is about a whole cache line. Trivial Python objects are barely smaller.
      • cogman10 1y ago
        The java GC approach is somewhat unique compared to other languages. There are multiple GCs and pretty much all of them are moving collectors. That means the JVM fairly rarely ends up freeing memory that it's claimed. A big spike will mean that it holds onto the spike's amount of memory. Many other GCed languages, such as swift, CPython, Go, do not use a moving collector. Instead, they allocate and pin memory and free it when not in use. The benefit to the JVM approach is heap allocations are wicked fast on pretty much all its collectors. Generally, to allocate it's a check to see if space is available and a pointer bump. For the other languages, you are bound to end up using a skiplist and/or arena allocator provided by your malloc implementation. Roughly O(log(n)) vs O(1) in performance terms. Don't get me wrong, the object header does eat a fair chunk of memory. Roughly double what another language will take. However, a lot of people confuse the memory which the JVM has claimed from the OS (and is thus reported by the OS) with the memory the JVM is actively using. 2 different things. It just so happens that for moving collectors like the JVM typically uses more reserved memory means fewer garbage collections and time spend garbage collecting.
  • loxs 1y ago
    The java runtime is a beast. Even the fact that another runtime is just capable to do a similar thing is impressive, disregard the fact that it might be better. Even being on par makes it interesting for me to maybe try it on my own.
  • javanonymous 1y ago
    The JVM tends to use as much memory as it can for performance reasons. It is not a reliable indicator of how much memory it actually needs. Why spend resources on clearing memory if there's still unused memory left? If memory is an issue, you can set a limit and the JVM will probably still work fine
    • cellularmitosis 1y ago
      If it were such an easy problem to fix, don’t you think they would have done so rather than rewriting in Swift?
      • cogman10 1y ago
        Having been in development for a long time, no. Frankly, it just takes some motivated senior devs and the tantalizing ability to put out the OP blog post and you've got something management will sign off on. Bonus points you get to talk about how amazing it was to use Apple tech to get the job done. I don't think they seriously approached this because the article only mentioned tuning G1GC. The fact is, they should have been talking about ZGC, AppCDS, and probably Graal if pause times and startup times were really that big a problem for them. Heck, even CRaC should have been mentioned. It is not hard to get a JVM to startup in sub second time. Here's one framework where that's literally the glossy print on the front page. [1] [1] https://quarkus.io/ https://quarkus.io/
        • jamesfinlayson 1y ago
          Yep, resume-driven development. I remember at a previous company a small group of people pushed a Go rewrite to speed everything up. The serious speed improvements came from re-architecting (elimination of a heavy custom framework, using a message queue instead of handling requests synchronously etc). They would have been better off fixing the original system so that everything could benefit from the improvements, not just the tiny bits that they carved off. Then the next annual report talked about improved scalability because of this amazing technology from Google.
• BonoboIO 1y ago
  Imagine what rust or go could have achieved
  • dontlaugh 1y ago
    Go is similar to Swift when it comes to mandatory costly abstractions. It’s only Rust (or C++, but unsafe) that have mostly zero-cost abstractions.
    • frizlab 1y ago
      Swift have them too now (non-Copyable types).
    • airspeedswift 1y ago
      Swift, Rust, and C++ all share the same underlying techniques for implementing zero-cost abstrations (primarily, fully-specialized generics). The distinction in Swift's case is that generics can also be executed without specialization (which is what allows generic methods to be called over a stable ABI boundary). Swift and Rust also allow their protocols to be erased and dispatched dynamically (dyn in Rust, any in Swift). But in both languages that's more of a "when you need it" thing, generics are the preferred tool.
      • dontlaugh 1y ago
        To an approximation, but stdlib and libraries will have a bias. In practice, abstractions in Rust and C++ more commonly are actually zero-cost than in Go or Swift. This is not a bad thing, I was just pointing out that Go doesn't have a performance advantage over Swift.
    • rafram 1y ago
      > It’s only Rust (or C++, but unsafe) that have mostly zero-cost abstractions. This just isn't true. It's good marketing hype for Rust, but any language with an optimizing compiler (JIT or AOT) has plenty of "zero-cost abstractions."
• Someone 1y ago
  My $0.02 is that Java not having value types (yet), while Swift has, is a large reason for the efficiency gains.
  • CharlieDigital 1y ago
    As a C# dev, I guess I've just taken it for granted that we have value types. Learned something new today (that Java apparently does not).
    • MBCook 1y ago
      It does for primitives. For user defined stuff we’ve recently gained records, which are a step in that direction, and a full solution is coming.
      • SigmundA 1y ago
        What about structs?
        • cogman10 1y ago
          I'm not sure with the semantics of structs for C#. What java is getting in the future is immutable data values where the reference is the value. When you have something like class Foo { int a; int b; } var c = new Foo(); in java, effectively the representation of `c` is a reference which ultimately points to the heap storage locations of `a, b`. In C++ terms, you could think of the interactions as being `c->b`. When values land, the representation of `c` can instead be (the JVM gets to decide, it could keep the old definition for various performance reasons) something like [type, a, b]. Or in C++ terms the memory layout can be analogous to the following: struct Foo { int a, int b }; struct Foo c; c.a = 1; c.b = 2;
          • CharlieDigital 1y ago
            This seems like C# `record struct` https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-10.0/record-structs https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...
• favorited 1y ago
  The post notes that the user-facing app was "introduced in the fall of 2024," so presumably the services aren't that legacy.
  • remus 1y ago
    You can learn a lot when writing V2 of a thing though. You've got lots of real world experience about what worked and what didn't work with the previous design, so lots of opportunity for making data structures that suit the problem more closely and so forth.
  • isodev 1y ago
    But did they write the backend from scratch or was it based on a number of “com.apple.libs.backend-core…” that tend to bring in repeating logic and facilities they have in all their servers? Or was it a PoC they promoted to MVP and now they’re taking time to rewrite “properly” with support for whatever features are coming next?
• misiek08 1y ago
  I love to always see such comments. On JVM you use crap like Spring and over-engineer everything. 20 types, interfaces and objects to keep single string in memory. JVM also like memory, but can be tailored to look okayish, still worse than opponents.
  • bdangubic 1y ago
    same - in every single language with incompetent team
  • paxys 1y ago
    And I'm 100% sure you can do the same in Swift.
    • jen20 1y ago
      Swift's limitations around reflection actually make it surprisingly difficult to create a typical Java-style mess with IOC containers and so forth.
      • pjmlp 1y ago
        Ever heard of Swift macros? Do you know where Java EE comes from? It started as an Objective-C framework, a language which Swift has full interoperability with. https://en.wikipedia.org/wiki/Distributed_Objects_Everywhere https://en.wikipedia.org/wiki/Distributed_Objects_Everywhere
        • jen20 1y ago
          > Ever heard of Swift macros? Yes, having lived on the daily build bleeding edge of Swift for several years, including while macros were being developed, I have indeed heard of them. > Do you know where Java EE comes from? Fully aware of the history. The point stands: it is substantially harder with Swift to make the kind of spring style mess that JVM apps typically become (of course there are exceptions: I typically suggest people write Java like Martin Thompson instead of Martin Fowler). Furthermore, _people just don’t do it_. I imagine you could visualise the percentage of swift server apps using an IOC container using no hands.
• rs186 1y ago
  Yes! The post would have been much more informative if it did an in-depth analysis of where the performance gain comes from. But Apple being Apple, I don't think they'll ever want to expose details on their internal systems, and we probably can only get such hand wavy statements.
  • MBCook 1y ago
    I suspect that didn’t fit into the goal of the blog post. I don’t think it’s meant to be a postmortem on figuring out what was going on and a solution, but more a mini white paper to point out Swift can be used on the server and has some nice benefits there. So the exact problems with the Java implementation don’t matter past “it’s heavy and slow to start up, even though it does a good job”.
• gt0 1y ago
  Agree, this is almost always where the benefits come from. You get to write v2 of the software with v1 to learn from.
• mpweiher 1y ago
  With the 90% reduction in memory consumption, I'd wager that most if not all the performance improvement came from that. In fact, it is a little surprising that hardware utilization only dropped 50%. Reduced memory consumption for cloud applications was apparently also the primary reason IBM was interested in Swift for a while. Most cloud applications apparently sit mostly idle most of the time, so the number of clients you can multiplex on single physical host is limited by memory consumption, not CPU throughput. And Java, with the JIT and the GC, has horrible memory consumption.
  • dgs_sgd 1y ago
    Interesting. If IBM was trying to solve for memory consumption, why do you think they picked Swift over alternatives that might also have achieved lower memory consumption?
    • dagmx 1y ago
      Swift is at a good middle ground of performance, safety and ease of use. It has higher level ergonomics that something like rust lacks (as much as I like rust myself), doesn’t have many of the pitfalls of Go (error handling is much better for example) and is relatively easy to pickup. It’s also in the same ballpark performance as rust or C++. It’s not perfect by any means, it has several issues but it’s quickly becoming my preferred language as well for knocking out projects.
      • ngcc_hk 1y ago
        What are those issues I wonder. I used for one project and not sure how deep the rabbit hole might be. Just wonder.
    • tgma 1y ago
      Which other sufficiently popular modern language that's more efficient than Java lacks a tracing GC? Rust and Swift are pretty much the only two choices and Rust is arguably much more pain in the ass for the average joe enterprise coder.
  • 1y ago
• tialaramex 1y ago
  Sure, maybe you can get money to have some businesses try out rewriting their line-of-business software in the same language versus in a different language and get some results. My expectation is that if you put the work in you can get actual hard numbers, which will promptly be ignored by every future person asking the same "question" with the same implied answer. If the "just rewrite it and it'll be better" people were as right as they often seem to believe they are, a big mystery is JWZ's "Cascade of Attention-Deficit Teenagers" phenomenon. In this scenario the same software is rewritten, over, and over, and over, yet it doesn't get faster and doesn't even fix many serious bugs.
  • staplers 1y ago
    If the "just rewrite it and it'll be better" people were as right as they often seem to believe Generally speaking, technological progress over thousands of years serves to validate this. Sure, in the short term we might see some slippage depending on talent/expertise, but with education and updated application of learnings, it's generally true.
  • tilne 1y ago
    For others that hadn’t head of CADT either: https://www.jwz.org/doc/cadt.html https://www.jwz.org/doc/cadt.html I confess to having been part of the cascade at various parts of my career.