4 ms·

Maybe these optimizations benefit the two users who do the operation three times a year. In such an extreme case no amount of optimization work would be profit

by lpapez 2y ago

Maybe these optimizations benefit the two users who do the operation three times a year. In such an extreme case no amount of optimization work would be profitable. So the parent comment asks a very valid question: how much total time was saved by this and who asked for it to be saved (paying or free tier customers for example)? People who see the business side of things rightfully fear when they hear the word "optimization", it's often not the best use of limited development resources - especially in an early stage product under development.

sroussey 2y ago
I do wish that when people write about optimization that they would then multiply by usage, or something similar. Another way is to show CPU usage over a fleet of servers before and after. And then reshuffle the servers and use fewer and use the number of servers no longer needed as the metric. Number of servers have direct costs, as well as indirect costs, so you can even derive a dollar value. More so if you have a growth rate.
- godelski 2y ago
  > I do wish that when people write about optimization that they would then multiply by usage, or something similar. How? You can give specific examples and then people make the same complaints because it isn't relevant to their use case. It's fairly easy to extrapolate the numbers to specific cases. We are humans, and we can fucking generalize. I'll agree there isn't much to the article, but I find this ask a bit odd. Do you not have all the information to make that calculation yourself? They should have done that if they're addressing their manager, but it looks like a technical blog where I think it is fair to assume the reader is technical and can make these extrapolations themselves.
  - sroussey 2y ago
    I hear you. However, I have one rule of writing: assume the reader is lazy. It is not that they are, but the assumption goes a long way in making content digestible. Also, I think knowing the combined effect is super interesting. For example, micro-benchmarks are fun to use and see improvements, but I also want to know the effect on the whole program.
    - godelski 2y ago
      I do wonder though if assuming the reader is lazy is the best. Especially in technical posts. I think there is a difficulty in balancing forcing the person to digest what you say and making it approachable (especially when you consider a noisy audience). It is a natural filter, is that good or bad? Guess depends. Agreed about the microbenchmarks and scale. Things don't always scale as expected. But I think there are a lot of variables here so it might be difficult to portray an accurate expected result. Though I can see this being worthwhile for anyone wanting to build RAGs or process lots of text into some embeddings. Also looks like the project is still under active development and started 6 months ago (single dev?) so I'm not sure we should expect to see too big of scale: https://github.com/bosun-ai/swiftide https://github.com/bosun-ai/swiftide So idk, that seems like exactly the kinda thing HN should be well suited for: new projects where people are hacking together useful frameworks. But idk, I guess if YC is funding companies who's business model is to fork an OSS then the bar might be lower than I think. But I thought we were supposed to be hackers (not necessarily crackers) ¯\_(ツ)_/¯
godelski 2y ago
> So the parent comment asks a very valid question: how much total time was saved by this and who asked for it to be saved (paying or free tier customers for example)? That is a hard question to answer because it very much depends on the use case, which is why I gave a vague response in my comment. Truth be told, __there is no answer__ BECAUSE it depends on context. In the case of AI agents, yeah, 50% is going to save you a ton of money. If you make LLM calls once a day, then no, probably not. Part of being the developer is to determine this tradeoff. Specifically, that's what technical managers are for, communicating technical stuff to business people (sure, your technical manager might not be technical, but someone being bad at their job doesn't make the point irrelevant, it just means someone else needs to do the job). You're right about early stage products, but there's lots of moderate and large businesses (and yes, startups) that don't optimize but should. Most software never optimizes and it has led to a lot of enshitification. Yes, move fast and break things, but go back and clean up, optimize, and reduce your tech debt, because you left a mess of broken stuff in your wake. But it is weird to pigeonhole to early stage startups.

by lpapez 2y ago

sroussey 2y ago

I do wish that when people write about optimization that they would then multiply by usage, or something similar. Another way is to show CPU usage over a fleet of servers before and after. And then reshuffle the servers and use fewer and use the number of servers no longer needed as the metric. Number of servers have direct costs, as well as indirect costs, so you can even derive a dollar value. More so if you have a growth rate.

godelski 2y ago
> I do wish that when people write about optimization that they would then multiply by usage, or something similar. How? You can give specific examples and then people make the same complaints because it isn't relevant to their use case. It's fairly easy to extrapolate the numbers to specific cases. We are humans, and we can fucking generalize. I'll agree there isn't much to the article, but I find this ask a bit odd. Do you not have all the information to make that calculation yourself? They should have done that if they're addressing their manager, but it looks like a technical blog where I think it is fair to assume the reader is technical and can make these extrapolations themselves.
- sroussey 2y ago
  I hear you. However, I have one rule of writing: assume the reader is lazy. It is not that they are, but the assumption goes a long way in making content digestible. Also, I think knowing the combined effect is super interesting. For example, micro-benchmarks are fun to use and see improvements, but I also want to know the effect on the whole program.
  - godelski 2y ago
    I do wonder though if assuming the reader is lazy is the best. Especially in technical posts. I think there is a difficulty in balancing forcing the person to digest what you say and making it approachable (especially when you consider a noisy audience). It is a natural filter, is that good or bad? Guess depends. Agreed about the microbenchmarks and scale. Things don't always scale as expected. But I think there are a lot of variables here so it might be difficult to portray an accurate expected result. Though I can see this being worthwhile for anyone wanting to build RAGs or process lots of text into some embeddings. Also looks like the project is still under active development and started 6 months ago (single dev?) so I'm not sure we should expect to see too big of scale: https://github.com/bosun-ai/swiftide https://github.com/bosun-ai/swiftide So idk, that seems like exactly the kinda thing HN should be well suited for: new projects where people are hacking together useful frameworks. But idk, I guess if YC is funding companies who's business model is to fork an OSS then the bar might be lower than I think. But I thought we were supposed to be hackers (not necessarily crackers) ¯\_(ツ)_/¯

godelski 2y ago

> So the parent comment asks a very valid question: how much total time was saved by this and who asked for it to be saved (paying or free tier customers for example)? That is a hard question to answer because it very much depends on the use case, which is why I gave a vague response in my comment. Truth be told, __there is no answer__ BECAUSE it depends on context. In the case of AI agents, yeah, 50% is going to save you a ton of money. If you make LLM calls once a day, then no, probably not. Part of being the developer is to determine this tradeoff. Specifically, that's what technical managers are for, communicating technical stuff to business people (sure, your technical manager might not be technical, but someone being bad at their job doesn't make the point irrelevant, it just means someone else needs to do the job). You're right about early stage products, but there's lots of moderate and large businesses (and yes, startups) that don't optimize but should. Most software never optimizes and it has led to a lot of enshitification. Yes, move fast and break things, but go back and clean up, optimize, and reduce your tech debt, because you left a mess of broken stuff in your wake. But it is weird to pigeonhole to early stage startups.