4 ms·

> What’s a little sad, is NPM should not be operating like a company with 1000+ employees. The “persuade us users want this” approach is only going to stop volu

> What’s a little sad, is NPM should not be operating like a company with 1000+ employees. The “persuade us users want this” approach is only going to stop volunteers. They should be proactively identifying efforts like this and helping you bring it across the finish line. Says who? Says an engineer? Says a product person? NPM is a company with 14 employees; with a system integrated into countless extremely niche and weird integrations they cannot control. Many of those integrations might make a professional engineer's hair catch fire - "it should never be done this way!" - but the real world is that the wrong way is the majority of the time. There's no guarantee that many of the downloads come from the official client, just as one example. The last thing they need, or I want, or any of their customers want, or their 14 employees need, is something that might break backwards compatibility in an extremely niche case, anger a major customer, cause countless support tickets, all for a tiny optimization nobody cares about. This is something I've learned here about HN that, for own mental health, I now dismiss: Engineers are obsessed with 2% optimizations here, 5% optimizations there; unchecked, it will literally turn into an OCD outlet, all for things nobody in the non-tech world even notices, let alone asks about. Just let it go.

liontwist 2y ago
Open source needs to operate differently than a company because people don’t have time/money/energy to deal with bullshit. Hell. Even 15 employees larping as a corporation is going to be inefficient. what you and NPM are telling us, is that they are happy to take free labor, but this is not an open source project. > Engineers are obsessed with 2% optimizations here Actually in large products these are incredible finds. But ok. They should have the leadership to know which bandwidth tradeoffs they are committed to and tell him immediately it’s not what they want, rather than sending him to various gatekeepers.
- gjsman-1000 2y ago
  Correct; NPM is not an "open source project" in the sense of a volunteer-first development model. Neither is Linux - over 80% of commits are corporate, and have been for a decade. Neither is Blender anymore - the Blender Development Fund raking in $3M a year calls the shots. Every successful "large" open source project has outgrown the volunteer community. > Actually in large products these are incredible finds. In large products, incredible finds may be true; but breaking compatibility with just 0.1% of your customers is also an incredible disaster.
  - liontwist 2y ago
    > breaking compatibility with just 0.1% Yes. But in this story nothing like that happened.
    - gjsman-1000 2y ago
      But NPM has no proof their dashboard won't light up full of corporate customers panicking the moment it goes to production; because their hardcoded integration to have AWS download packages and decompress them with a Lambda and send them to an S3 bucket can no longer decompress fast enough while completing other build steps to avoid mandatory timeouts; just as one stupid example of something that could go wrong. IT is also demanding now that NPM fix it rather than modify the build pipeline which would take weeks to validate, so corporate's begging NPM to fix it by Tuesday's marketing blitz. Just because it's safe in a lab provides no guarantee it's safe in production.
      - liontwist 2y ago
        Ok, but why is the burden on him to show that? Are they not interested in improving bandwidth and speed for their users? The conclusion of this line of reasoning is to never make any change. If contributions are not welcome, don’t pretend they are and waste my time. > can no longer decompress fast enough Already discussed this in another thread. It’s not an issue.
      - maccard 2y ago
        That’s an argument against making any change to the packaging system ever. “It might break something somewhere” isn’t an argument, it’s a paralysis against change. Improving the edge locality of delivery of npm packages could speed up npm installs. But speeding up npm installs might cause the CI system which is reliant on it for concurrency issues to have a race condition. Does that mean that npm can’t ever make it faster either?
        gjsman-1000 2y ago
        It is an argument. An age old argument: "If it ain't broke, don't fix it."
stonemetal12 2y ago
NPM is a webservice. They could package the top 10-15 enhancements call it V2. When 98% of traffic is V2 turn V1 off. Repeat every 10 years or so until they work their way into having a good protocol.
maccard 2y ago
> Engineers are obsessed with 2% optimizations here, 5% optimizations there; unchecked, it will literally turn into an OCD outlet, all for things nobody in the non-tech world even notices, let alone asks about. Just let it go. I absolutely disagree with you. If the world took more of those 5% optimisations here and there everything would be faster. I think more people should look at those 5% optimisations. In many cases they unlock knowledge that results in a 20% speed up later down the line. An example from my past - I was tasked with reducing the running speed of a one shot tool we were using at $JOB. It was taking about 15 minutes to run. I shaved off seconds here and there with some fine grained optimisations, and tens of seconds with some modernisation of some core libraries. Nothing earth shattering but improvements none the less. One day, I noticed a pattern was repeating and I was fixing an issue for the third time in a different place (searching a gigantic array of stuff for a specific entry). I took a step back and realised that if I replaced the mega list with a hash table it might fix every instance of this issue in our app. It was a massive change, touching pretty much every file. And all of a sudden our 15 minute runtime was under 30 seconds. People used this tool every day, it was developed by a team of engineers wildly smarter than me. But it had grown and nobody really understood the impact of the growth. When it started that array was 30, 50 entries. On our project it was 300,000 and growing every day. Not paying attention to these things causes decay and rot. Not every change should be taken, but more people should care.
lyu07282 2y ago
> Says who? > Says an engineer? I prevent cross-site scripting, I monitor for DDoS attacks, emergency database rollbacks, and faulty transaction handlings. The Internet heard of it? Transfers half a petabyte of data every minute. Do you have any idea how that happens? All those YouPorn ones and zeroes streaming directly to your shitty, little smart phone day after day? Every dipshit who shits his pants if he can't get the new dubstep Skrillex remix in under 12 seconds? It's not magic, it's talent and sweat. People like me, ensuring your packets get delivered, un-sniffed. So what do I do? I make sure that one bad config on one key component doesn't bankrupt the entire fucking company. That's what the fuck I do.

liontwist 2y ago

Open source needs to operate differently than a company because people don’t have time/money/energy to deal with bullshit. Hell. Even 15 employees larping as a corporation is going to be inefficient. what you and NPM are telling us, is that they are happy to take free labor, but this is not an open source project. > Engineers are obsessed with 2% optimizations here Actually in large products these are incredible finds. But ok. They should have the leadership to know which bandwidth tradeoffs they are committed to and tell him immediately it’s not what they want, rather than sending him to various gatekeepers.

gjsman-1000 2y ago
Correct; NPM is not an "open source project" in the sense of a volunteer-first development model. Neither is Linux - over 80% of commits are corporate, and have been for a decade. Neither is Blender anymore - the Blender Development Fund raking in $3M a year calls the shots. Every successful "large" open source project has outgrown the volunteer community. > Actually in large products these are incredible finds. In large products, incredible finds may be true; but breaking compatibility with just 0.1% of your customers is also an incredible disaster.
- liontwist 2y ago
  > breaking compatibility with just 0.1% Yes. But in this story nothing like that happened.
  - gjsman-1000 2y ago
    But NPM has no proof their dashboard won't light up full of corporate customers panicking the moment it goes to production; because their hardcoded integration to have AWS download packages and decompress them with a Lambda and send them to an S3 bucket can no longer decompress fast enough while completing other build steps to avoid mandatory timeouts; just as one stupid example of something that could go wrong. IT is also demanding now that NPM fix it rather than modify the build pipeline which would take weeks to validate, so corporate's begging NPM to fix it by Tuesday's marketing blitz. Just because it's safe in a lab provides no guarantee it's safe in production.
    - liontwist 2y ago
      Ok, but why is the burden on him to show that? Are they not interested in improving bandwidth and speed for their users? The conclusion of this line of reasoning is to never make any change. If contributions are not welcome, don’t pretend they are and waste my time. > can no longer decompress fast enough Already discussed this in another thread. It’s not an issue.
    - maccard 2y ago
      That’s an argument against making any change to the packaging system ever. “It might break something somewhere” isn’t an argument, it’s a paralysis against change. Improving the edge locality of delivery of npm packages could speed up npm installs. But speeding up npm installs might cause the CI system which is reliant on it for concurrency issues to have a race condition. Does that mean that npm can’t ever make it faster either?
      - gjsman-1000 2y ago
        It is an argument. An age old argument: "If it ain't broke, don't fix it."

stonemetal12 2y ago

NPM is a webservice. They could package the top 10-15 enhancements call it V2. When 98% of traffic is V2 turn V1 off. Repeat every 10 years or so until they work their way into having a good protocol.

maccard 2y ago

> Engineers are obsessed with 2% optimizations here, 5% optimizations there; unchecked, it will literally turn into an OCD outlet, all for things nobody in the non-tech world even notices, let alone asks about. Just let it go. I absolutely disagree with you. If the world took more of those 5% optimisations here and there everything would be faster. I think more people should look at those 5% optimisations. In many cases they unlock knowledge that results in a 20% speed up later down the line. An example from my past - I was tasked with reducing the running speed of a one shot tool we were using at $JOB. It was taking about 15 minutes to run. I shaved off seconds here and there with some fine grained optimisations, and tens of seconds with some modernisation of some core libraries. Nothing earth shattering but improvements none the less. One day, I noticed a pattern was repeating and I was fixing an issue for the third time in a different place (searching a gigantic array of stuff for a specific entry). I took a step back and realised that if I replaced the mega list with a hash table it might fix every instance of this issue in our app. It was a massive change, touching pretty much every file. And all of a sudden our 15 minute runtime was under 30 seconds. People used this tool every day, it was developed by a team of engineers wildly smarter than me. But it had grown and nobody really understood the impact of the growth. When it started that array was 30, 50 entries. On our project it was 300,000 and growing every day. Not paying attention to these things causes decay and rot. Not every change should be taken, but more people should care.

lyu07282 2y ago

> Says who? > Says an engineer? I prevent cross-site scripting, I monitor for DDoS attacks, emergency database rollbacks, and faulty transaction handlings. The Internet heard of it? Transfers half a petabyte of data every minute. Do you have any idea how that happens? All those YouPorn ones and zeroes streaming directly to your shitty, little smart phone day after day? Every dipshit who shits his pants if he can't get the new dubstep Skrillex remix in under 12 seconds? It's not magic, it's talent and sweat. People like me, ensuring your packets get delivered, un-sniffed. So what do I do? I make sure that one bad config on one key component doesn't bankrupt the entire fucking company. That's what the fuck I do.