6 ms·
Is there a torrent or dump of all the images? I wouldn't mind seeding / helping to keep an archive of all this. This is pretty great!
by urda 10y ago
Is there a torrent or dump of all the images? I wouldn't mind seeding / helping to keep an archive of all this. This is pretty great!
- gregsadetsky 10y agoTo scrape, start here http://metmuseum.org/api/collection/collectionlisting?offset=0&pageSize=0&perPage=100&sortBy=Relevance&sortOrder=asc http://metmuseum.org/api/collection/collectionlisting?offset... and increase 'offset' by 100. The JSON output contains image URLs. The total number of results is 441048, so finding another endpoint that doesn't enforce a limit of 100 on the 'perPage' argument would be great. --- EDIT: thanks to spitfare, I updated the perPage argument to 100. The site doesn't allow larger values, but that's definitely a great start! (about 4k requests to get everything)
- deleted 10y ago[deleted]
- scosman 10y agoThey published an index on Github. It would likely be a more responsible way to access this versus hammering their API. https://github.com/metmuseum/openaccess https://github.com/metmuseum/openaccess
- gregsadetsky 10y agoUnfortunately, as stated in that repo's readme: `Images are not included and are not part of the dataset.` The repo doesn't include the images, which is understandable; however, the CSV file doesn't even include links to the images.
- kjell 10y agoExcluding the images themselves from the dataset is one thing, but there's an open issue to include a link to the images: https://github.com/metmuseum/openaccess/issues/2 https://github.com/metmuseum/openaccess/issues/2
- DenisM 10y agoAre there any docs for this API? I wish I could filter by artist...
- lostmsu 10y agoIf you finish scraping, can you still make a .torrent?
- awqrre 10y agoedit: it doesn't download the highest available resolution... This Perl script should get most of the images (but it would probably be better to save them in the original sub-directories): use strict; use warnings; use LWP::Simple; use JSON qw(from_json); my $url = "http://metmuseum.org/api/collection/collectionlisting?offset="; my $args = "&pageSize=0&perPage=100&sortBy=Relevance&sortOrder=asc"; my $offset = 0; while ($offset < 5000) { my $decoded = from_json(get($url.$offset.$args)); $offset = $offset + 100; my @results = @{ $decoded->{'results'} }; foreach my $i ( @results ) { my $filename = $i->{"largeImage"}; my $title = $i->{"title"}; print "Status: ".getstore("http://images.metmuseum.org/CRDImages/".$filename,$title.".jpg")." ".$filename. "\n"; } }
- y04nn 10y agoCan't agree more, this should be shared on torrent, it would make it more accessible. The torrent would need to include the information on the art piece.
- kjell 10y agoHere's an example torrent of metadata and images from the Rijksmuseum put together by an art history researcher: http://matthewlincoln.net/2015/10/19/the-rijksmuseum-as-bittorrent.html http://matthewlincoln.net/2015/10/19/the-rijksmuseum-as-bitt...
- ConceptJunkie 10y agoI just finished downloading this this morning. I've been able to look through some of it so far and there's some really fascinating stuff. I'm particularly interested in the paintings for use as wallpapers, but there are lots of printed materials dating back to the 17th century (and probably earlier).
- mdlincoln 10y agoI'm working on pulling the images now, like I did for the Rijksmuseum CC0 dump. FWIW a good place to host that torrent is the Internet Archive - it's great for discoverability.
- lostmsu 10y agoHow did it go?