7 ms·
Microsoft TerraServer (1998)
- corywatilo 13y ago"A terabyte holds 250 full-length movies. It is a lot of data." =]
- eksith 13y agoMy, how times change! Come to think of it, a terabyte is a lot of data still (as is a gigabyte). Just not for movies these days though.
- MichaelGG 13y agoMost torrents at 720p (and seemingly fine quality to me) come in around a GB. So it's enough for almost 1000 movies.
- RobAtticus 13y agoGood quality 720p should definitely be more than a gig for movies. Even high quality 480p rips came in at over a gig for most movies. I'm sure the torrents are watchable, but there are sacrifices being made somewhere.
- dzhiurgis 13y agoOr 10 movies in 4K resolution.
- mentat 13y agoThis was so much fun to use. I built perl scripts around this (learning at the same time differences between lat/lon and UTM) to map wireless access points[0][1]. 0. http://wigle.net/phpbb/viewtopic.php?t=9 http://wigle.net/phpbb/viewtopic.php?t=9 1. http://wigle.net/gps/gps/main/download/ http://wigle.net/gps/gps/main/download/
- eksith 13y agoThank you for sharing this. Do you have a snapshot archive since you first started this? I wonder how much the landscape has changed since then.
- mentat 13y agoI can see if there's anything in any of my backups. Nowadays I'd just use Google Earth's history feature (optionally feeding in time based KMLs of access points). I think the WiGLE interface supports time filtering, but I don't know if that's officially exposed.
- eksith 13y agoThanks! It probably isn't exposed, but it would be nice to see these grow over the years. It may even expose other things like how community hubs have shifted and what demographics are being catered to. Overlays from census data or other insights might reveal some interesting things.
- epoxyhockey 13y agoWiGLE, just a little bit. I still wonder what happened to blackwave. :)
- mentat 13y agoYeah, I've been wondering about that for a few years. After he disappeared, none of the Irvine people would even talk about it.
- CervezaPorFavor 13y agoYet another entry in the list of missed opportunities. But I suppose it's easy to criticise people with the benefit of hindsight.
- jaggederest 13y agoI don't think it's wrong to criticize them for missing this as an opportunity. I remember playing with it at the time, it was really revolutionary. There was literally nothing like this available to the general public before this. And who owns mapping these days? Google. It's pretty sad that Microsoft wasn't able to turn a six year head start into a reasonable business.
- santoshalper 13y agoThat's sort of been the story of Microsoft for the past 15 years or so, lots of cool ideas, then a failure to execute and make a great product out of them. I wonder if young engineers these days realize how formidable Microsoft used to be. There was a time when having MSFT in your rearview mirror meant you were in deep shit. Now, they're just another big IT company.
- dhotson 13y agoCheck out MapBox if you haven't already. There's still some room for competition. :-)
- MichaelGG 13y agoYou think that's sad? MSN Messenger had millions of users, complete with their "friends". And they failed to turn that into any social network worth anything. Duh. (Same with Skype.)
- jgreen10 13y agoMicrosoft has a culture of being first of the market, though not always with the best product. Products like Windows 98 and IE4 were buggy, but so far ahead of their time that they had a crushing impact and it took the rest of the world years to catch up. Nowadays competitors with better solutions are usually not far behind. Google and Apple are rarely first to the market, but usually the best. It's a more effective strategy nowadays. Amazon is one of the few companies that still manages to be first to the market and be successful (Kindle, AWS). Perhaps because many of the old Microsofties went there.
- fsckin 13y agoI dug around a bit and found a research paper[0] about TerraServer. It's got all sorts of neat details not in this link. TerraServer hardware configuration parameters: Max hits per day 40 million/day Max SQL queries per day 37 million/day Max image downloads / day 35 million/day Bandwidth to Internet 200 Mbps = 2 Terabytes/day Concurrent web connections = 6,000 connections Web front ends: Six 4-way 200 Mhz Compaq Proliant 5500, .5GB ram Database back-end 1 8-way 440Mhz Compaq AlphaServer 8400 10GB ram, 3.2 TB raid5 324 9GB Ultra SCSI disks 324 x 9GB disks for 2.9TB? You could store 1.2PB with the same number of 4TB disks. Doesn't seem like Moore's Law is holding up for storage, nor for internet speeds. [0] http://www.math.uaa.alaska.edu/~afkjm/cs401/project_fall_04/terrarep.pdf http://www.math.uaa.alaska.edu/~afkjm/cs401/project_fall_04/...
- frozenport 13y agoI wonder if 324 on Lustre would have more throughput than a 1 4TB disk?
- enigmo 13y agoDefinitely... and _way_ more IOPS.
- fsckin 13y agoThey would be leaps and bounds faster. Especially comparing random IOPS. Let's just assume they were using a Seagate Cheetah 9LP. That's a 9GB, 10k RPM, 1MB cache UW SCSI disk that was widely available in 1998, for a whopping... $1200 each. $390,000 worth of disks... wow. That could buy 1300 * 4TB SAS drives today with a total raw storage of 5.2PB! 324 * 9LP's would have around 5.5GB/sec of total read/write throughput. That easily beats a single 4TB disk with around 200MB/sec read/write performance and terrible IOPS in comparison to even a single 9LP. SSDs compare much more favorably here. 10 x 1TB SSDs for ~$6500 could easily reach 5.5GB/sec throughput and triple the capacity of the 324 x 9GB drives.
- mariusz79 13y agoYou're comparing off-the-shelf disks to enterprise-class storage.
- ryanburk 13y ago> Tom Barclay, Robert Eberl, Jim Gray, John Nordlinger, Guru Raghavendran, Don Slutz, Greg Smith, Phil Smoot Microsoft Research and Development a number of those folks went on to build some impressively large scale services. hotmail, skydrive, etc. I wish jim was still around...
- andyjohnson0 13y agoWired article from 2007 on the use of satellite imaging in the search for Jim Gray. http://www.wired.com/techbiz/people/magazine/15-08/ff_jimgray?currentPage=all http://www.wired.com/techbiz/people/magazine/15-08/ff_jimgra...
- rob-alarcon 13y agoI didn't know about this, is a Sad story.
- wiradikusuma 13y agoReading "..combining five terabytes of image data.." and "Figure 1. The TerraServer hardware" while looking at the portable 2TB external HDD sitting on my desk... how technology advances rapidly. Makes me wonder how (storage) technology can be in 10-15 years.
- mariusz79 13y agoYet our smartphones can't seem to get over 64GB limit. Personally, I'm afraid that the cloud will effectively slow-down the progress in the storage space, at least in the consumer portion of it.
- rtkwe 13y agoThere's space inside for more storage in most phones. 64GB is just a point where the number of people who would actually use that amount of space rapidly diminishes.
- akgerber 13y agoSD cards are available with up to 256GB of storage. They just cost $400 or so[0], so I don't think enough people see the value of paying for that much storage in a phone. [0]http://www.amazon.com/Lexar-Professional-256GB-Memory-LSD256CTBNA600/dp/B0090BEWKY http://www.amazon.com/Lexar-Professional-256GB-Memory-LSD256...
- mariusz79 13y agoWhat about the phones that come without sd card support?
- akgerber 13y agoThe flash memory chips used in phones are essentially the same as those used in SD cards, and phones often use SDIO for internal communication. The SD card is just a point of reference for the cost and availablity of flash memory unbundled from a phone.
- codeulike 13y agoI remember this. I think at the time they said just by turning the thing on they were doubling the amount of internet-accessible data in the world.
- chris_wot 13y agoFunny how things change a bit. EMC now owns Clarion. DEC no longer exists. EMC now owns Legato Nerworker. Seagate's backup management group was bought by Veritas, who then merged with Symantec. Such is the potted history of storage companies!
- bnolsen 13y agoThe company I worked for back ended some of these datasets. Definitely a bunch of usgs imagery. At the time several racks of solaris servers were required to serve up the imagery using tomcat for the requests. Those machines were in canada. Windows was used for arcgis and oracle. Shortly after opteron was released and ibm came out with a 64bit jvm I moved the whole thing except for the esri portion over to a simple load balancer running apache and two mirrored dual opterons running fedora 64bit (2002). A total half rack which handled about a million image reqs a day or so. Each req involved decomposing, reading and stitching image mosaics together and sending out in under 10s. Unfortunately we were back ending more than just terraserver so I don't remember who had what specs. With all the memory alloc/dealloc going on inside the java process we were fortunate if it didn't require a full restart every 12 hours. The funniest part of the whole thing was that I rewrote the engine from java over to a c++ stitcher and got request times under 1s on a dual p3 600. That was bypassing the esri server, oracle database and java stuff. At that time the umn mapserver started getting interesting as well although their image processing was glacial at the beginning. And then I was retasked and I left before being able to finish up and deploy.
- codeulike 13y agoWait, so you're saying that the Terraserver project that was supposed to show Windows and SQL Server capabilities was back-ended by a third party company using tomcat, oracle, java and linux. That doesn't make any sense. The OP-linked paper describes how it was all done with NT, IIS and SQL Server. http://msdn.microsoft.com/en-us/library/aa226316%28v=sql.70%29.aspx#msterra_hardware http://msdn.microsoft.com/en-us/library/aa226316%28v=sql.70%...
- dmead 13y agoi was so confused about the big deal with google maps when this had been around for like 7 years.
- estebank 13y agoI remember both. Google maps was by far much more enjoyable to use. Just like between Hotmail vs GMail.
- bluedino 13y ago10GB in that Alpha server seems like a massive amount of memory at the time. But DEC claimed capability of up to 28GB. An 8400 configured the way they had it was probably a half a million dollars, not including the disks of course. You can see how far behind Intel hardware was at the time - the quad-CPU Pentium Pro boxes only had 256MB of memory! Sure, that was only a $40k box from Compaq... Fun fact: At the time, Quake engine licensees were buying 4-way Alpha and Intel servers to do the level processing for first person shooters.
- zmonkeyz 13y agoI remember trying to visit the maps on this thing way back in the day. It was always broke.