3 ms·
I'm kinda a little disappointed at the answers provided here. 1) I downloaded a 49.6 MB TIFF file from here: http://hubblesite.org/newscenter/archive/releases/
by samptemp 12y ago
I'm kinda a little disappointed at the answers provided here.
1) I downloaded a 49.6 MB TIFF file from here:
http://hubblesite.org/newscenter/archive/releases/2004/32/image/d/warn/ http://hubblesite.org/newscenter/archive/releases/2004/32/im...
2) From the OSX terminal:
$ time sips -Z 150 hs-2004-32-d-full_tif.tif
Resulting file size is 150x150px and 32KB.
Time to run process:
1.48s user 0.15s system 98% cpu 1.663 total
The superpower computer required for this incredible speed is a 3 year old MacBook Pro (with SSD) ;-)
UPDATE:
I created 10 copies of the image. Total size: 520.1 MB
$ time sips -Z 150 .tif
Time to process 10 images:
14.98s user 1.44s system 94% cpu 17.384 total
(approximately 1.498 per image).
2000000 images 1.663 seconds = 38 days.
Note: This is using larger images (49.6MB vs the 25-30MB images you have) and is using a single MacBook Pro. Divide amongst a few machines and be done in a week. GraphicsMagick could possibly be even faster.
ANOTHER UPDATE:
Since your files are smaller (20-30MB), I found a 30MB jpg sample here:
http://sto-rvlt-01.sys.comcast.net/speedtest/random4000x4000.jpg http://sto-rvlt-01.sys.comcast.net/speedtest/random4000x4000...
Time to process was much faster:
0.37s user 0.04s system 97% cpu 0.418 total
2000000 images * 0.418 seconds = 9.67 days
At that speed we are talking under 10 days. On a single 3 year old MacBook Pro. Using built-in software. Without any optimizations. Find 5 computers in the NY Public Library and you'll be done in 2 days.
I am not saying this is the fastest solution. I posted this because it appears that people are over-engineering this problem or proposing solutions which will cost a lot of money (Amazon, bandwidth, shipping hard drive fees, etc).
- samptemp 12y agoTo add to my above comment: a big high-five to those offering their free time or pro bono advice to the New York Public Library! You guys really rock!
- jcanyc 12y agoIn terms of cost/expense, AWS is very cost effective for this type of work load. If you build out the skill set and scripts to run this type of job, you can process the images, store them on S3 and kill the running infrastructure. You can then run this process for big and small workloads as the needs arise on demand without having to own servers or space in a data center.