3 ms·

Getting a 53% performance boost on a 20+ year old codebase by running a bunch of experiments is pretty exciting to me: https://github.com/Shopify/liquid/pull/20

by simonw 8mo ago

Getting a 53% performance boost on a 20+ year old codebase by running a bunch of experiments is pretty exciting to me: https://github.com/Shopify/liquid/pull/2056 https://github.com/Shopify/liquid/pull/2056

discreteevent 8mo ago
Developers make these kinds of improvements all the time. Are you saying that it would have been impossible without AI?
- simonw 8mo ago
  That codebase existed for 20 years and had contributions from nearly 200 people. Sure, they could have come up with those optimizations without AI... but they didn't. What's your theory for why that is?
  - skydhash 8mo ago
    Maybe because it’s a non issue. I saw that those improvements are in the order of micro seconds, while the transfer time of a page is measure in 1/10 seconds or even several seconds. Even a game engine have something like 15 ms to have a frame ready (60hz).
    - simonw 8mo ago
      Lots of small improvements add up - the total performance improvement is 53%. That's significant. If you're the size of Shopify that represents a huge saving in server costs and improved customer-facing latency.
      - skydhash 8mo ago
        > the total performance improvement is 53%. That's significant. This percentage is meaningless on its own. It’s 4 ms shaved off a 7 ms process. You would need to time a whole flow (and I believe databases would add a lot to it, especially with network latency) and figure out how significant the performance improvement is actually. And that without considering if the code changes is not conflicting with some architectural change that is being planned.
        simonw 8mo ago
        I'll take a 53% performance boost in my template language any day of the week.
gzread 8mo ago
It's written by AI, so those numbers are probably fake. What numbers do you get on the benchmark?
- simonw 8mo ago
  I see no reason to run my own benchmark here, the numbers aren't run through an LLM they're right there in the JSONL file: https://github.com/Shopify/liquid/blob/3182b7c1b3758b0f5fe2d0fcc71a48bbcb11c946/autoresearch.jsonl https://github.com/Shopify/liquid/blob/3182b7c1b3758b0f5fe2d...

by simonw 8mo ago

discreteevent 8mo ago

Developers make these kinds of improvements all the time. Are you saying that it would have been impossible without AI?

simonw 8mo ago
That codebase existed for 20 years and had contributions from nearly 200 people. Sure, they could have come up with those optimizations without AI... but they didn't. What's your theory for why that is?
- skydhash 8mo ago
  Maybe because it’s a non issue. I saw that those improvements are in the order of micro seconds, while the transfer time of a page is measure in 1/10 seconds or even several seconds. Even a game engine have something like 15 ms to have a frame ready (60hz).
  - simonw 8mo ago
    Lots of small improvements add up - the total performance improvement is 53%. That's significant. If you're the size of Shopify that represents a huge saving in server costs and improved customer-facing latency.
    - skydhash 8mo ago
      > the total performance improvement is 53%. That's significant. This percentage is meaningless on its own. It’s 4 ms shaved off a 7 ms process. You would need to time a whole flow (and I believe databases would add a lot to it, especially with network latency) and figure out how significant the performance improvement is actually. And that without considering if the code changes is not conflicting with some architectural change that is being planned.
      - simonw 8mo ago
        I'll take a 53% performance boost in my template language any day of the week.

gzread 8mo ago

It's written by AI, so those numbers are probably fake. What numbers do you get on the benchmark?

simonw 8mo ago
I see no reason to run my own benchmark here, the numbers aren't run through an LLM they're right there in the JSONL file: https://github.com/Shopify/liquid/blob/3182b7c1b3758b0f5fe2d0fcc71a48bbcb11c946/autoresearch.jsonl https://github.com/Shopify/liquid/blob/3182b7c1b3758b0f5fe2d...