2 ms·

Fun fact, earliy versions of lemmy were using my open source image hosting and postprocesing cdn (with S3,NFS,ftp.. storage) PictShare[1] until the devs raised

Fun fact, earliy versions of lemmy were using my open source image hosting and postprocesing cdn (with S3,NFS,ftp.. storage) PictShare[1] until the devs raised an issue[2] about a huge memory leak when resizing this[3] 10mb PNG file which caused the memory to raise over 1.5 gigabytes and basically crashed the backend. So they ditched PictShare and wrote their own tool[4] only to realize it has the exact same problem because of the way some PNG files are stored basically any resize operation causes huge RAM spikes no matter which software or implementation is used. [1] https://github.com/HaschekSolutions/pictshare https://github.com/HaschekSolutions/pictshare [2] https://github.com/HaschekSolutions/pictshare/issues/120 https://github.com/HaschekSolutions/pictshare/issues/120 [3] https://commons.wikimedia.org/wiki/File:Pizigani_1367_Chart_10MB.jpg https://commons.wikimedia.org/wiki/File:Pizigani_1367_Chart_... [4] https://github.com/LemmyNet/lemmy/pull/807 https://github.com/LemmyNet/lemmy/pull/807

ntoskrnl 4y ago
That image is already 300MB of uncompressed pixel data. That's a hard floor on the memory usage unless you do something crazy like build a streaming image processing pipeline.
- btdmaster 4y ago
  With ImageMagick convert, it does take up 1.5 gigabyte of RAM and take quite about 5 seconds. With ffmpeg, it only takes up 150MB and finishes about 10 times faster! I am really curious what makes swscaler from ffmpeg so memory efficient compared to other tools.
antifa 4y ago
Also PictShare supports S3 compatible backends, wasn't cool to see lemmy drop that so frivolously.