5 ms·
This is what containers solve. Don't waste time manually installing things. Store state in a database via the app on a different host.
by sneak 3y ago
This is what containers solve. Don't waste time manually installing things. Store state in a database via the app on a different host.
- ElectricalUnion 3y agoI used to think that the 300MB containers I shipped to prd were reasonably unoptimized. Those AMD ROCm containers are like 14GiB compressed.
- kunwon1 3y agoSpeaking as someone who has encountered similar difficulties, this response has strong 'Draw the rest of the owl' vibes
- sneak 3y agoSpeaking as someone who has solved these difficulties hundreds of times, "draw the rest of the owl" doesn't tell you the specific things to google to get detailed examples and tutorials on how millions of others have sidestepped these repeated issues.
- fennecfoxy 3y ago[dead]
- machinawhite 3y agoWell then you're wasting time managing your containers? Have you ever used k8s, it's a full time job lol
- zztop44 3y agoI don’t think anyone is suggesting k8s for running an ai model as a side project on a single machine.
- machinawhite 3y ago> a side project on a single machine Sorry I don't see how will containers help here
- itake 3y agoYep... you spend hours messing around with docker containers and debugging all the weird build errors. I am less familiar with storing data in a db (for ml hosting concerns), but I'd imagine it would add overhead (as opposed to accessing files on disk). You also have to deal with hosting a db and configuring the schema.
- sneak 3y agoYou "spend hours messing around" with everything you don't know or understand at first. One could say the same about writing the software itself. At its core Dockerfiles are just shell scripts with worse syntax, so it's not really that much more to learn. Once you get it done once, you don't have to screw around with it anymore, and you have it on any box you want in seconds. In either case you have to spend hours screwing around with your environment. If those hours result in a Dockerfile, then it's the last time. If they don't, then it's each time you want it on a new host (which as was correctly pointed out a pain in the ass). Storing data in a database vs in files on disk is like application development 101 and is pretty much a required skill period. It's required that you learn how to do this because almost all applications revolve around storing some kind of state and, as was noted, you can't reasonably expect it to persist on the app server without additional ops headaches. Many people will host dbs for you without you having to think about it. Schema is only required if you use a structured db (which is advisable) but it doesn't take that long.
- ElectricalUnion 3y ago> Storing data in a database vs in files on disk is like application development 101 It's OK until you're dealing with, say, 130GiB of tensors, on what is effectively a binary blob that needs to be mostly in VRAM somehow. I really don't want to read 130GiB of blobs from a database all the time.
- novok 3y agoIMO tensors & other large binary blobs are fairly edge-casey. You might as well treat them like video files and video file servers also don't store large videos in databases either, and most devs don't have large binary blob management experience.