3 ms·
Here's some stuff I have done in the past year, I work for a small company, but run a personal computing cluster of 167 servers that I pay for out of my own poc
by malux85 12y ago
Here's some stuff I have done in the past year, I work for a small company, but run a personal computing cluster of 167 servers that I pay for out of my own pocket. I really enjoy loading "big" datasets into them and working on improving algorithms or gaining insight into the data.
I (try and) network around London and offer my services for free to people who have interesting problems.
- Very high resolution FMRI data. A single scan can be 10-20GB
- Infringing URLs for a piracy company, 4 billion rows
- DNA sequences and Protein Data, lots of variation in sizes, from a few hundred MB's of string data, to hundreds of GBs
- RAW radio data for a military skunkworks project (10's of GB / min)
I would really like to find an investor who could take me off my full time job, I have 3 quite large projects I would like to build, one I have almost finished.
- sysk 12y agoDo you actually have 167 servers running in your home or do you rent them on e.g. EC2 when you need them? If it's the former case, is it because it makes financial sense (I'd be surprised) or is it for the experience/fun?
- malux85 12y agoI have 3 blade servers in my house, these act as command and control machines, the 167 are rented on EC2
- yankoff 12y agoWhat do you pay for such cluster if I may ask? I assume it's not in EC2? Have you thought of adding a web-crawling service on the top of that?
- malux85 12y agoIt is on EC2. I pay about £2500 a month - but the monthly cost is lowered because a lot of the machines are on reserved instances - with a big chunk paid up front (sometimes by the people I work with) ... I usually work for free because I'm not really interested in financial gain but more interested in interesting problems ... but I gotta pay the bills sometimes, and the people I work with are usually more than happy to chip in if I need it :)