7 ms·

Challenge yourself to use stdlib only for a while, or the future. It sounds unrealistic and I'm going to get flamed, but hear me out. It works. Most of my deve

by aliqot 4y ago

Challenge yourself to use stdlib only for a while, or the future. It sounds unrealistic and I'm going to get flamed, but hear me out. It works. Most of my development these days is in reasonably complete languages like go, rust, zig, and various scripting languages, so your mileage may vary if you're writing in something like rune, hare or carbon that is still taking shape. If you think I'm crazy but have a lingering skepticism, challenge yourself to spend one day, one week, or one month using the stdlib only. If that is unrealistic in your setting a compromise could be to only use libraries that were created by yourself. You wont come out of it as an enlightened samurai monk with a celebrity HN presence, but you will gain an immense sense of scrutiny that didn't exist before as you take a look at the libraries you want to use after that. For those who think this is just 100% bull, consider how we survived before google, youtube, and stackexchange existed.

pclmulqdq 4y ago
I am thinking very hard about the CAP theorem right now while working on a billing system for a cloud API right now and it is an absolute joy. No, it won't deploy in version 1, 2, or 3, but it might in version 4, and if it does, it will be glorious. You can find cool technical problems anywhere as long as you are willing to take the path less traveled.
- TeMPOraL 4y ago
  > You can find cool technical problems anywhere as long as you are willing to take the path less traveled. After doing that a few times, I'm no longer sure if the reward of tackling cool problems to create more robust, better, faster components is worth the stress of missing deadlines. Looking back on what value of better work materializes and what is, per YAGNI, usually wasted, I just had a thought: perhaps the right way is to take the easy/dumb way and focus all available time/effort to optimize it for performance - instead of abstraction and extensibility. Because in my experience, nobody ever extends the code the way you envisioned - if they do it at all, they do it by first refactoring it to suit their own idea. And, nobody ever goes back to fix performance. Therefore, making things abstract and extensible is mostly a wasted work - but making things fast pays back for as long as the code is in use.
  - pclmulqdq 4y ago
    I am doing this for a particular purpose, though, that no billing system I have seen has. I hate metered API billing, and I don't want to use it. In particular, I don't want a customer running up a $10,000 bill and calling me for a refund. It probably will cost me O($1,000) to give them a refund, between the processing fees and the lost compute, and I will be out that money. Most companies that would ask for a refund also won't pay the bill when it arrives (which is, I think, why AWS is so liberal with refunds). Instead, I want to do credit-based billing: you buy credits, and when you get to 0 credits, you are cut off (with an auto-refill option for the "metered billing experience," but with strict spending limits). This is, in my opinion, a much better UX than metered billing. From a distributed systems perspective, it's isometric to "metered billing with a hard spending cap" which may be the ultimate version. The problem with credit-based billing (and why nobody does it) is that if you have a service in multiple datacenters, you have to consistently update a database to make sure that you don't drop below 0 credits, and that is very slow. However, by fiddling with the CAP theorem the way CockroachDB/Spanner do, I think we can do credit-based billing and make it feel like the AP system that metered billing is, and behave like a CP system only when we absolutely need to. Also, this is basically all enabled by the fact that AWS has a precise time service. In theory, version 1 of the API will be only in one region, version 2 will have active/passive redundancy, and around version 3 or 4 I want to switch to active/active in several DCs to give low latency and high reliability.
    - Dunedan 4y ago
      > I hate metered API billing, and I don't want to use it. > Instead, I want to do credit-based billing: you buy credits, and when you get to 0 credits, you are cut off (with an auto-refill option for the "metered billing experience," but with strict spending limits). This is, in my opinion, a much better UX than metered billing. How do your customers feel about that? Have you researched if your customers are comfortable spending their money upfront to buy credits they might only use much later (if ever)? For small amounts of money that might not be a big deal, but if we're talking about thousands of dollars that might look different.
kspacewalk2 4y ago
I'm not gonna flame you, but I will note that, as someone who gets paid to use my judgement to decide on the optimal trade-off between quality, time spent on the project, and its future maintainability... I feel like all three will suffer quite a bit with this self imposed "handicap".
- uoaei 4y ago
  This is the main crux of the issue IMO: feature output velocity. With the enforcement of sprint-scale development scope, you really don't have time to iterate on a wide-reaching and supportive base layer of software infrastructure so you reach for tools that will get you what you need within the timeframe demanded by whoever hired you.
  - Hermitian909 4y ago
    As someone who's been fortunate enough to work on and lead these kinds of projects (and watch coworkers work on them) I've come to a near opposite conclusion which is that sprints reveal how bad the ROI on this work tends to be. The way many of these projects go is someone very smart works with subject matter experts to map out the problem space. The smart person (or people) then go and begin building this set of primitives and integrating the product, adjusting as they go along accruing some warts. After 6-12 months we have this beautiful tool that improves developer velocity, new features are easy to code; then disaster strikes. It turns out the map of the problem space was wrong! A bunch of things the team believed to be invariants aren't! Suddenly business needs are forcing developers to tear down walls in their beautiful abstraction castle until all that's left is a tangled maze no other developer has a hope of understanding. Now the project is a millstone around the dev team's neck rather than velocity boost they'd hoped for. The way these projects more often succeed is some senior engineer pastes together an abstraction layer sends it out into the world. It gets heavily abused for years until finally the team says "We know this sucks, and there's a lot of business value if we make it nice, let's invest a bunch of time in retrofitting this" and fight like hell to make the business case. IMO this tends to lead to better projects and value (though unfortunately many companies make the fight harder than it should be)
tayo42 4y ago
I don't think I you can compare the internet of the 90s to today's internet. The expectations and scale are no where similar.
jesse__ 4y ago
Going one step further, challenge yourself to _not_ use stdlib, just raw language constructs. Need a hashtable? Write one. You won't come out with a celebrity HN presence, but you may gain enlightened samurai monk status.
- spoils19 4y ago
  Going one step further, challenge yourself not to use raw language constructs, but dive deep into assembly. Need a conditional? Time to jump around. I wrote a web server that scales to over 1 million requests per second, and I found out it's much more maintainable, scalable and environmentally friendly for our company.
  - danparsonson 4y ago
    That sounds like it was a lot of fun but I'm wondering what assembly gave you that was not available in e.g. C? My understanding is that most compilers can out-optimise the average developer, so are you an above-average developer (well I guess you are) or did assembly enable you to do something that was difficult in a higher level language? Or was it more about the challenge (which I'm totally on board with by the way)?
    - strangetortoise 4y ago
      > My understanding is that most compilers can out-optimise the average developer Do you happen to know where (what source) you got that from? I'm genuinely curious, as to my knowledge, compilers are generally still easily fooled by things that causes them unable to vectorize code or factor out conditional jumps. To give an example here, see Mike Actons' talk below (from about 43:10 onwards) in which he describes the compiler failures. https://youtu.be/rX0ItVEVjHc?t=2590 https://youtu.be/rX0ItVEVjHc?t=2590
      - danparsonson 4y ago
        No specific source I could point to really, just that I've been hanging around message boards like this for a long time. I didn't mean to say it's impossible to beat a compiler or they don't have any blind spots, but thanks for the link - sounds interesting :-)
Conscat 4y ago
Challenge yourself to not even use stdlib once in awhile. There are some interesting insights to glean about how much room for improvement we have even at the very bottom. https://youtu.be/BrBb0mqoIAc https://youtu.be/BrBb0mqoIAc
mikewarot 4y ago
I wrote entire systems with Turbo Pascal, and then Delphi out of the box. Many others did the same with Visual Basic 6, or the Microsoft Office 2000 suite with VBA, before the .NET infection took hold and Microsoft lost it's mind. All before Google, Youtube and Stack Exchange

by aliqot 4y ago

pclmulqdq 4y ago

I am thinking very hard about the CAP theorem right now while working on a billing system for a cloud API right now and it is an absolute joy. No, it won't deploy in version 1, 2, or 3, but it might in version 4, and if it does, it will be glorious. You can find cool technical problems anywhere as long as you are willing to take the path less traveled.

TeMPOraL 4y ago
> You can find cool technical problems anywhere as long as you are willing to take the path less traveled. After doing that a few times, I'm no longer sure if the reward of tackling cool problems to create more robust, better, faster components is worth the stress of missing deadlines. Looking back on what value of better work materializes and what is, per YAGNI, usually wasted, I just had a thought: perhaps the right way is to take the easy/dumb way and focus all available time/effort to optimize it for performance - instead of abstraction and extensibility. Because in my experience, nobody ever extends the code the way you envisioned - if they do it at all, they do it by first refactoring it to suit their own idea. And, nobody ever goes back to fix performance. Therefore, making things abstract and extensible is mostly a wasted work - but making things fast pays back for as long as the code is in use.
- pclmulqdq 4y ago
  I am doing this for a particular purpose, though, that no billing system I have seen has. I hate metered API billing, and I don't want to use it. In particular, I don't want a customer running up a $10,000 bill and calling me for a refund. It probably will cost me O($1,000) to give them a refund, between the processing fees and the lost compute, and I will be out that money. Most companies that would ask for a refund also won't pay the bill when it arrives (which is, I think, why AWS is so liberal with refunds). Instead, I want to do credit-based billing: you buy credits, and when you get to 0 credits, you are cut off (with an auto-refill option for the "metered billing experience," but with strict spending limits). This is, in my opinion, a much better UX than metered billing. From a distributed systems perspective, it's isometric to "metered billing with a hard spending cap" which may be the ultimate version. The problem with credit-based billing (and why nobody does it) is that if you have a service in multiple datacenters, you have to consistently update a database to make sure that you don't drop below 0 credits, and that is very slow. However, by fiddling with the CAP theorem the way CockroachDB/Spanner do, I think we can do credit-based billing and make it feel like the AP system that metered billing is, and behave like a CP system only when we absolutely need to. Also, this is basically all enabled by the fact that AWS has a precise time service. In theory, version 1 of the API will be only in one region, version 2 will have active/passive redundancy, and around version 3 or 4 I want to switch to active/active in several DCs to give low latency and high reliability.
  - Dunedan 4y ago
    > I hate metered API billing, and I don't want to use it. > Instead, I want to do credit-based billing: you buy credits, and when you get to 0 credits, you are cut off (with an auto-refill option for the "metered billing experience," but with strict spending limits). This is, in my opinion, a much better UX than metered billing. How do your customers feel about that? Have you researched if your customers are comfortable spending their money upfront to buy credits they might only use much later (if ever)? For small amounts of money that might not be a big deal, but if we're talking about thousands of dollars that might look different.

kspacewalk2 4y ago

I'm not gonna flame you, but I will note that, as someone who gets paid to use my judgement to decide on the optimal trade-off between quality, time spent on the project, and its future maintainability... I feel like all three will suffer quite a bit with this self imposed "handicap".

uoaei 4y ago
This is the main crux of the issue IMO: feature output velocity. With the enforcement of sprint-scale development scope, you really don't have time to iterate on a wide-reaching and supportive base layer of software infrastructure so you reach for tools that will get you what you need within the timeframe demanded by whoever hired you.
- Hermitian909 4y ago
  As someone who's been fortunate enough to work on and lead these kinds of projects (and watch coworkers work on them) I've come to a near opposite conclusion which is that sprints reveal how bad the ROI on this work tends to be. The way many of these projects go is someone very smart works with subject matter experts to map out the problem space. The smart person (or people) then go and begin building this set of primitives and integrating the product, adjusting as they go along accruing some warts. After 6-12 months we have this beautiful tool that improves developer velocity, new features are easy to code; then disaster strikes. It turns out the map of the problem space was wrong! A bunch of things the team believed to be invariants aren't! Suddenly business needs are forcing developers to tear down walls in their beautiful abstraction castle until all that's left is a tangled maze no other developer has a hope of understanding. Now the project is a millstone around the dev team's neck rather than velocity boost they'd hoped for. The way these projects more often succeed is some senior engineer pastes together an abstraction layer sends it out into the world. It gets heavily abused for years until finally the team says "We know this sucks, and there's a lot of business value if we make it nice, let's invest a bunch of time in retrofitting this" and fight like hell to make the business case. IMO this tends to lead to better projects and value (though unfortunately many companies make the fight harder than it should be)

tayo42 4y ago

I don't think I you can compare the internet of the 90s to today's internet. The expectations and scale are no where similar.

jesse__ 4y ago

Going one step further, challenge yourself to _not_ use stdlib, just raw language constructs. Need a hashtable? Write one. You won't come out with a celebrity HN presence, but you may gain enlightened samurai monk status.

spoils19 4y ago
Going one step further, challenge yourself not to use raw language constructs, but dive deep into assembly. Need a conditional? Time to jump around. I wrote a web server that scales to over 1 million requests per second, and I found out it's much more maintainable, scalable and environmentally friendly for our company.
- danparsonson 4y ago
  That sounds like it was a lot of fun but I'm wondering what assembly gave you that was not available in e.g. C? My understanding is that most compilers can out-optimise the average developer, so are you an above-average developer (well I guess you are) or did assembly enable you to do something that was difficult in a higher level language? Or was it more about the challenge (which I'm totally on board with by the way)?
  - strangetortoise 4y ago
    > My understanding is that most compilers can out-optimise the average developer Do you happen to know where (what source) you got that from? I'm genuinely curious, as to my knowledge, compilers are generally still easily fooled by things that causes them unable to vectorize code or factor out conditional jumps. To give an example here, see Mike Actons' talk below (from about 43:10 onwards) in which he describes the compiler failures. https://youtu.be/rX0ItVEVjHc?t=2590 https://youtu.be/rX0ItVEVjHc?t=2590
    - danparsonson 4y ago
      No specific source I could point to really, just that I've been hanging around message boards like this for a long time. I didn't mean to say it's impossible to beat a compiler or they don't have any blind spots, but thanks for the link - sounds interesting :-)

Conscat 4y ago

Challenge yourself to not even use stdlib once in awhile. There are some interesting insights to glean about how much room for improvement we have even at the very bottom. https://youtu.be/BrBb0mqoIAc https://youtu.be/BrBb0mqoIAc

mikewarot 4y ago

I wrote entire systems with Turbo Pascal, and then Delphi out of the box. Many others did the same with Visual Basic 6, or the Microsoft Office 2000 suite with VBA, before the .NET infection took hold and Microsoft lost it's mind. All before Google, Youtube and Stack Exchange