7 ms·
"Batteries included" ecosystems are the only persistent solution to the package manager problem. If your first party tooling contains all the functionality you
by bob1029 7mo ago
"Batteries included" ecosystems are the only persistent solution to the package manager problem.
If your first party tooling contains all the functionality you typically need, it's possible you can be productive with zero 3rd party dependencies. In practice you will tend to have a few, but you won't be vendoring out critical things like HTTP, TCP, JSON, string sanitation, cryptography. These are beacons for attackers. Everything depends on this stuff so the motivation for attacking these common surfaces is high.
I can literally count on one hand the number of 3rd party dependencies I've used in the last year. Dapper is the only regular thing I can come up with. Sometimes ScottPlot. Both of my SQL providers (MSSQL and SQLite) are first party as well. This is a major reason why they're the only sql providers I use.
Maybe I am just so traumatized from compliance and auditing in regulated software business, but this feels like a happier way to build software too. My tools tend to stay right where I left them the previous day. I don't have to worry about my hammer or screw drivers stealing all my bitcoin in the middle of the night.
- junon 7mo agoThis is a rather superlative and tunnel vision, "everything is a nail because I'm a hammer" approach. The truth is this is an exceedingly difficult problem nobody has adequately solved yet.
- bbkane 7mo agoI think the AI tooling is, if not completely solving sandboxing, at least making the default much better by asking you every time they want to do something and providing files to auto-approve certain actions. Package managers should do the same thing
- hectdev 7mo agoAnother layer of AI tooling is the cost of spinning up your own version of some libraries is lowered and can be made hyper specific to your needs rather than pulling in a whole library with features you'll never use.
- lelanthran 7mo ago> Another layer of AI tooling is the cost of spinning up your own version of some libraries is lowered and can be made hyper specific to your needs rather than pulling in a whole library with features you'll never use. Tell me about it. Using AI Chatbots (not even agents), I got a MVP of a packaging system[1] to my liking (to create packages for a proprietary ERP system) and an endpoint-API-testing tool, neither of which require a venv or similar to run. ------------------------------ [1] Okay, all it does now is create, sign, verify and unpack packages. There's a roadmap file for package distribution, which is a different problem.
- nailer 7mo ago> at least making the default much better by asking you every time they want to do something Really? I thought 'asking you every time they want to do something' was called 'security fatigue' and generally considered to be a bad thing. Yes you can concatenate files in the current project, Claude.
- mrsmrtss 7mo agoFully agree with this! I think today .NET is probably the most batteries included platform you can get. This means that even if you use third-party libraries, these typically depend only on first-party dependencies, making it much less likely for something shady to sneak in.
- Imustaskforhelp 7mo agoTo me, I really like Golang's batteries included platform. I am not sure about .NET though
- jeswin 7mo agoAnd now with NativeAOT, you can use C# like go - you don't need to ship the CLR.
- raddan 7mo agoWith the notable exception of cross-platform audio.
- zdc1 7mo agoThe other thing that keeps coming up is the github-code-is-fine-but-the-release-artifact-is-a-trojan issue. It really makes me question if "packages" should even exist in JavaScript, or if we could just be importing standard plain source code from a git repo. I understand why this doesn't work well with legacy projects, but it's something that the language could strive towards.
- embedding-shape 7mo ago> I understand why this doesn't work well with legacy projects, but it's something that the language could strive towards. Why wouldn't that work well with legacy projects? In fact, the projects I was a part of that I'd call legacy nowadays, was in fact built by copy-and-pasting .js libraries into a "vendor/" directory, and that's how we shipped it as well, this was in the days before Bower (which was the npm of frontend development back in the day), vendoring JS libs was standard practice, before package managers became used in frontend development too. Not sure why it wouldn't work, JavaScript is a very moldable language, you can make most things work one way or another :)(
- crab_galaxy 7mo ago`vendor/` folders give me the worst developer PTSD :p 6 conflicting versions of jquery, and you know every single one of them was monkey patched, cemented into the codebase forever.
- EMM_386 7mo agoThis might make things worse not better. Yes - the postinstall hook attack vector goes away. You can do SHA pinning since Git's content addressing means that SHA is the hash of the content. But then your "lockfile" equivalent is just... a list of commit SHAs scattered across import statements in your source? Managing that across a real dependency tree becomes a nightmare. This is basically what Deno's import maps tried to solve, and what they ended up with looked a lot like a package registry again. At least npm packages have checksums and a registry that can yank things.
- 7mo ago
- gib444 7mo agoWhat kind of apps do you build / industry etc?
- christophilus 7mo agoHonestly, you can get pretty far with just Bun and a very small number of dependencies. It’s what I love most about Bun. But, I do agree with you generally. .NET is about as good as I’ve ever seen for being batteries included. I just hate the enterprisey culture that always seems to pervade .NET shops.
- wongarsu 7mo ago> In practice you will tend to have a few, but you won't be vendoring out critical things like HTTP, TCP, JSON, string sanitation, cryptography Unless you are Python, where the standard library includes multiple HTTP libraries and everyone installs the requests package anyways. Few languages have good models for evolving their standard library, so you end up with lots of bad designs sticking around forever. Libraries are much easier to evolve, giving them the advantage in terms of developer UX and performance.
- ptx 7mo agoI'm pretty sure it's really one HTTP library: urllib.request is built on top of http.client. But the very Java-inspired API for the former is awful.
- paintbox 7mo agoWhat type of developer chooses UX and performance over security? So reckless. I removed the locks from all the doors, now entering/exiting is 87% faster! After removing all the safety equipment, our vehicles have significantly improved in mileage, acceleration and top speed!
- wongarsu 7mo agoBetter developer UX can directly lead to better safety. "You are holding it wrong" is a frequent source of security bugs, and better UX reduces the ways you can hold it wrong, or at least makes you more likely to hold it the right way
- skydhash 7mo agoFriction is helpful. Putting seatbelts on takes more time than just driving, but it’s way safer for the driver. Current dev practices increase speed, not safety.
- lelanthran 7mo ago> Better developer UX can directly lead to better safety. Depends. If you had to add to a Makefile for your dependencies, you sure as hell aren't going to add 5k dependencies manually just to get a function that does $FOO; you'd write it yourself. Now, with AI in the mix, there's fewer and fewer reasons to use so many dependencies.
- invaliduser 7mo agoFor a lot of code, I switched to generating code rather than using 3rd party libraries. Things like PEG parsers, path finding algorithms, string sanitizers, data type conversion, etc are very conveniently generated by LLMs. It's fast, reduces dependencies, and feels safer to me.
- troad 7mo agoAh, so you've traded the possibility of bad dependencies for certainty.
- raddan 7mo agoRemember, our objective function here is “feels safe.”
- invaliduser 7mo agoHow can you come to that conclusion, given the specific examples I have given, which are tedious to write, but easy to proof-read and test?
- deleted 7mo ago[deleted]
- deleted 7mo ago[deleted]
- senordevnyc 7mo agoBecause AI threatens the identity of many programmers.
- deleted 7mo ago[deleted]
- deleted 7mo ago[deleted]
- troad 7mo agoWhat are some examples of batteries-included languages that folk around here really feel productive in and/or love? What makes them so great, in your opinion? (Leaving aside thoughts on language syntax, compile times, tooling etc - just interested in people's experiences with / thoughts on healthy stdlibs)
- pyjarrett 7mo agoThese are the big ones I use, specifically because of the standard libraries: Python (decent standard library) - It's pretty much everywhere. There's so many hidden gems in that standard library (difflib, argparse, shlex, subprocess, cmd) C#/F# (.NET) C# feels so productive because of how much is available in .NET Core, and F# gets to tag along and get it all for free too. With C# you can compile executables down to bundle the runtime and strip it down so your executables are in the 15 MiB range. If you have dotnet installed, you can run F# as scripts.
- troad 7mo agoThese are definitely some good thoughts, thanks! Do you worry at all about the future of F#? I've been told it's feeling more and more like a second-class citizen on .NET, but I don't have much personal experience.
- pyjarrett 7mo agoI used to, but the knowledge of .NET seems mostly transferrable to C#. It's super useful to do `dotnet fsi` and then work out the appropriate .NET calls in the F# repl.
- troad 7mo agoThat's a really good point, thank you. I'm running Elixir in prod right now, so F# would be up my alley. As someone with all the memory of a fruit fly, I love a good REPL when I'm trying to recall the various magic incantations that make the computer go.
- 7mo ago
- gedy 7mo ago> "Batteries included" ecosystems are the only persistent solution to the package manager problem. The irony in this case is that axios is not really needed now given that fetch is part of the JS std lib.
- afavour 7mo agoIrony is that Node has no need for Axios, native fetch support has been there for years, so in terms of network requests it is batteries included.
- deleted 7mo ago[deleted]
- pier25 7mo agoPeople use axios or ky because with fetch you inevitably end up writing a small wrapper on top of it anyway.
- afavour 7mo agoSome might say the tradeoff of writing a small wrapper is worth it given what’s been demonstrated here.
- pier25 7mo agoYeah but what about other deps like db drivers?
- jmull 7mo agoIn my experience people feel the need to wrap axios too.
- kube-system 7mo agoThese are the kind of people I hope AI replaces
- nathancahill 7mo agoAI was trained on Axios wrappers, so it's just going to be wrappers all the way down. Look inside any company "API Client" and it's just a branded wrapper around Axios.
- TZubiri 7mo agoBut javascript is batteries included in this case, you can use xmlhttprequest or fetch
- pier25 7mo agoI agree. Got downvoted a lot the other day for proposing Node should solve fundamental needs.
- raincole 7mo agoDifferent programmers have very different ideas about what is "all the functionality you typically need."
- Lord_Zero 7mo agoSo, youre on Microsoft then, judging by ScottPlot you write .NET desktop apps. If you use Dapper, you probably use Microsoft.Data.SqlClient, which is... distributed over NuGet and vulnerable to supply chain attack. You may not need many deps as a desktop dev. Modern day line of business apps require a lot more deps. CSVHelper, ClosedXML, AutoMapper, WebOptimizer, NetEscapades.AspNetCore.SecurityHeaders. Yes less deps people need the better but it doesn't fix trhe core problem. Sharing and distrib uting code is a key tenant of being able to write modern code.
- tliltocatl 7mo agoI agree that dependencies are a liability, but, sadly, "batteries included" didn't work out for Python in practice (i. e. how do I even live without numpy? No, array aren't enough).
- jcgl 7mo agoTo the extend that Python is indeed "batteries included," that seems true. But just how "batteries included" is it? I'd argue that its batteries are pretty limited. Exhibit A: everybody uses the third-party requests instead of the stdlib urllib. Exhibit B: http.server isn't a production-ready webserver, so people use Flask or something beefier. I'd contrast Python with Go, which has an amazing stdlib for the domains that Go targets. This last part is key--Go has a more focused scope than Python, and that makes it easier for its stdlib to succeed.
- seunosewa 7mo agoWe could have different Python package bundles: Python base. Python webdev. Python desktop.
- 12_throw_away 7mo ago> http.server isn't a production-ready webserver, so people use Flask [...] Nit, but relevant nit: Flask is also not a production-grade webserver. You could say it is also missing batteries ... and those batteries are often missing batteries too. Which is why you don't deploy flask, you deploy flask on top of gunicorn on top of nginx. It's missing batteries all the way down (or at least 3 levels down).
- jcgl 7mo agoAppreciate the nit. Had no idea that Flask wasn't production-grade. Yeesh. I really don't miss this part of the Python world. When I started on backend stuff ~10 years ago, the morass of runtime stuff for Python webservers felt bewildering. uWSGI? FastCGI? Gunicorn? Twisted? Like you say, missing batteries all the way down, presumably due to async/GIL related pains. Then you step into the Go world and it's just the stdlib http package. Anyway, ranting aside, batteries included is a real thing, and it's great. Python just doesn't have it.
- philipwhiuk 7mo agoLanguage churn makes this problem worse. Frankly inventing a new language is irresponsible these days unless you build on-top of an existing ecosystem because you need to solve all these problems.
- cozzyd 7mo agoor you don't use a package manager where anyone can just publish a package (i.e. use your system package manager). There is still some risk, but it is much smaller. Like, if xz were distributed by PyPI or NPM, everyone would have been pwned, but instead it was (barely) found. It's true that system repos doesn't include everything, but you can create your own repositories if you really need to for a few things. In practice Fedora/EPEL are basically sufficient for my needs. Right now I'm deploying something with yocto, which is a bit more limited in slection, but it's pretty easy to add my own packages and it at least has hashes so things don't get replaced without me noticing (to be fair, I don't know if the security practices of open-embedded recipes are as strong as Fedora...).
- calvinmorrison 7mo agoit's muddying what a package is. A package, or a distro, is the people who slave and labor over packaging, reviewing, deciding on versions to ship, having policies in place, security mailing lists, release schedules, etc. just shipping from npm crap is essentially the equivelant of running your production code base against Arch AUR pkgbuilds.
- dec0dedab0de 7mo agoBatteries included systems are still susceptible to supply chain attacks, they just move slower so it’s not as attractive of a target. I think packages of a certain size need to be held to higher standards by the repositories. Multiple users should have to approve changes. Maybe enforced scans (though with trivy’s recent compromise that wont be likely any time soon) Basically anything besides lone developer can decide to send something out on a whim that will run on millions of machines.
- Drakim 7mo agoWhile technically true, it's so much slower that it's essentially a different thing. Third party packages being attacked is a near daily occurrence. First party attacks happens on the timescale and frequency of decades. It's like the difference in protecting your home from burglars and foreign nation soldiers. Both are technically invaders to your home, but the scope is different, and the solutions are different.
- xienze 7mo ago> they just move slower so it’s not as attractive of a target. Well, there’s other things. Maven doesn’t allow you to declare “version >= x.y.z” and doesn’t run arbitrary scripts upon pulling dependencies, for one thing. The Java classpath doesn’t make it possible to have multiple versions of the same library at the same time. That helps a lot too. NPM and the way node does dependency management just isn’t great. Never has been.
- binsquare 7mo agoyep! This is exactly the world I'm working towards with packaging tooling with a virtual machine i.e. electron but with virtual machines instead so the isolation aspect comes by default.
- deleted 7mo ago[deleted]
- commandlinefan 7mo ago> "Batteries included" ecosystems are the only persistent solution Or write your own stuff. Yes, that's right, I said it. Even HTTP. Even cryptography. Just because somebody else messed it up once doesn't mean nobody should ever do it. Professional quality software _should_ be customized. Professional developers absolutely can and should do this and get it right. When you use a third-party HTTP implementation (for example), you're invariably importing more functionality than you need anyway. If you're just querying a REST service, you don't need MIME encoding, but it's part of the HTTP library anyway because some clients do need it. That library (that imports all of its own libraries) is just unnecessary bloat, and this stuff really isn't that hard to get right.
- iknowstuff 7mo ago> Even cryptography Good luck
- lelanthran 7mo ago> When you use a third-party HTTP implementation (for example), you're invariably importing more functionality than you need anyway. If you're just querying a REST service, you don't need MIME encoding, but it's part of the HTTP library anyway because some clients do need it. That library (that imports all of its own libraries) is just unnecessary bloat, and this stuff really isn't that hard to get right. This post is modded down (I think because of the "roll your own crypto vibe", which I disagree with), but this is actually spot on the money for HTTP. The surface area for HTTP is quite large, and your little API, which never needed range-requests, basic-auth, multipart form upload, etc suddenly gets owned because of a vulnerability in one of those things you not only never used, you also never knew existed! "Surface area" is a problem, reducing it is one way to mitigate.
- commandlinefan 7mo ago> the "roll your own crypto vibe", which I disagree with Again, you run into the attack surface area here. Think about the Heartbleed vulnerability. It was a vulnerability in the DTLS implementation of OpenSSL, but it affected every single user, including the 99% that weren't using DTLS. Experienced developers can, and should, be able to elide things like side-channel attacks and the other gotchas that scare folks off of rolling their own crypto. The right solution here is better-defined, well understood acceptance criteria and test cases, not blindly trusting something you downloaded from the internet.
- rhdunn 7mo agoThere are several issues with "Batteries Included" ecosystems (like Python, C#/.NET, and Java): 1. They are not going to include everything. This includes things like new file formats. 2. They are going to be out of date whenever a standard changes (HTML, etc.), application changes (e.g. SQLite/PostgreSQL/etc. for SQL/ORM bindings), or API changes (DirectX, Vulcan, etc.). 3. Things like data structures, graphics APIs, etc. will have performance characteristics that may be different to your use case. 4. They can't cover all nice use cases such as the different libraries and frameworks for creating games of different genres. For example, Python's XML DOM implementation only implements a subset of XPath and doesn't support parsing HTML. The fact that Python, Java, and .NET have large library ecosystems proves that even if you have a "Batteries Included" approach there will always be other things to add.
- CommonGuy 7mo agoWhy would they be out of date? The ecosystems themselves (for example .NET) receives regular updates. Yes, they cannot include everything, but enough that you do not _need_ third party packages.
- Groxx 7mo ago"Batteries included" means "ossification is guaranteed", yah. "stdlib is where code goes to die" is a fairly common phrase for a reason. There's clearly merit to both sides, but personally I think a major underlying cause is that libraries are trusted. Obviously that doesn't match reality. We desperately need a permission system for libraries, it's far harder to sneak stuff in when doing so requires an "adds dangerous permission" change approval.
- EGreg 7mo agoNot at all. We simply need M-of-N auditors to sign off on major releases of things. And the package managers need to check this (the set of auditors can be changed, same as browser PKI for https) before pulling things down. That's the system we have in our Safebox ecosystem
- jaikechen 7mo agoI think stay vigilant is better than trust anything.
- fireant 7mo agoWhile it's true that the packages are first party, .NET still relies on packages to distribute code that's not directly inside the framework. You still probably transiently depend on `Microsoft.Extensions.Hosting.Abstractions ` for example - if the process for publishing this package was compromised, you'd still get owned.
- rendaw 7mo agoThis just moves the trust from one group to another. Now the standard library/language maintainers need to develop/maintain more high quality software. So either they get overworked and burn out, don't address issues, fail to update things or they recruit more people who need to be trusted. Then they are responsible for doing the validation that you should have done. Are they better equipped to do that? Maybe they go, oh hey, Axios is popular and widely trusted, let's make it an official library and bring the maintainers into the fold... wait isn't this exactly where we started? What process did you trust the standard library/language maintainers in the first place? How do they differ from any other major library vendor?
- resonious 7mo agoI agree with you and follow the same principles myself, but JavaScript already has HTTP, and yet everyone still uses Axios. So the problem isn't that JS doesn't have batteries, it's that people don't want to use them for some reason. I'm guessing it's similar to the tragedy of the commons phenomenon. When things are freely available people tend to overuse or carelessly use them. NPM is just too easy to use. If a package offers a 1% ergonomics increase over a builtin function, many folks will just go for it because it costs them nothing (well, it seems to cost them nothing).