3 ms·
This origin story is at least a century too late. If you visit the sorting exhibit at the computer history museum, you will rapidly discover that big data and
by hedora 8y ago
This origin story is at least a century too late.
If you visit the sorting exhibit at the computer history museum, you will rapidly discover that big data and data science predates computers.
Their oldest machine sticks punch cards above a pool of mercury, and then drops a bunch of metal pins on to the card.
The signals from the metal pins activated mechanical counters.
In this way, it could perform aggregation queries for operators with constant-sized sufficient statistics (~= map only jobs). it was used by the US census bureau.
Later machines could sort the punch cards, which enabled map reduce queries.
The term “data base” came about 50 years after the census machine was in use.
A military installation was running the equivalent of map reduce queries and needed a noun to describe their mountains of punch cards (or maybe reel to reel tapes).
- jds375 8y agoDo you have any further references about what exact military installation was doing this and when? I’m reading Cryptonomicon by Neal Stephenson and one of the main characters builds a mercury-based data analysis machine that reads punch cards to analyze communication data in WWII and this sounds eerily similar.
- gumby 8y agosee my reply to the parent post
- triplee 8y agoA lot of that book took into account real life events and people. Unfortunately I don't have specifics on that one, so hopefully the parent does. Also, that book is fantastic, as are the prequels.
- bane 8y agoIIR mercury-based memories part of a larger class of memory called "delay-line memory" that dates all the way back the 1920s. The basic premise is that it uses the time delay of signals propagating in some medium to temporarily "store" a value. The modern analog is the need to refresh DRAM in order to keep the bits flipped the right way. https://en.wikipedia.org/wiki/Delay_line_memory https://en.wikipedia.org/wiki/Delay_line_memory
- wefarrell 8y agoI point to the discovery behind the source of the broad street cholera epidemic in 1854 as an example of sophisticated pattern recognition that predates computers: https://en.wikipedia.org/wiki/1854_Broad_Street_cholera_outbreak#Broad_Street_outbreak https://en.wikipedia.org/wiki/1854_Broad_Street_cholera_outb...
- gumby 8y agoBack when the Computing-Tabulating-Recording company expanded and renamed themselves International Business Machines their bread and butter was automated card reading and managing, but I know the manual card reading process (with rods and trays) continued, amazingly, into the 1950s at least (I had professors who had used those techniques as grad students). Hollerith, however, already had electrical (I hesitate to say "electronic:) card reading in the 19th century (Hollerith was one of the progenitors of CTR by the way.). All those cool optimizations like mercury-bath readers, vacuum columns for tapes etc were simply expensive performance optimizations of tasks previously done mechanically, only viable for people with huge datasets. By the 1940s the Bell long distance area code routing tables were stored as sets of metal punch cards the size of North American automobile license plates. As they were metal (they were read a lot) they were presumably read mechanically though I haven't ever seen the apparatus. Yes, they updated the routing tables by sending huge crates of metal tablets to each toll office. Whippersnappers might not remember the ubiquity of punch cards (actually I don't remember either, and I'm the oldest of the genXers), but apparently you got one with your bills, like utilities and charge card bills, so they could assign payment to the correct account. I do remember highway tolls being collected by issuing a punch card upon entry to the turnpike and then having the toll collector use that to calculate the toll due when you exited. Apparently the phrase "do not fold, spindle or mutilate" was a commonly used tagline for jokes about the computerization of society as all theses cards bore that legend.