2 ms·
A year ago I wrote a half-assed map/reduce in PHP. I partitioned a day's worth of logs across hundreds of gzipped CSV files by key hash, so I could run four red
by prodigal_erik 17y ago
A year ago I wrote a half-assed map/reduce in PHP. I partitioned a day's worth of logs across hundreds of gzipped CSV files by key hash, so I could run four reducers (the box has four cores) and each could suck a partition into 1/4 of physical memory.
After about six months of pathological random access (and what I assume to be epic fragmentation), the disk failed. I hope to migrate to our shiny new Hadoop cluster while the replacement disk lasts.