4 ms·

I'll answer your question and then engage in some shameless self-promotion. I wrote about it after posting my comment. Here's the chunk which answers your quest

by itgoon 17y ago

I'll answer your question and then engage in some shameless self-promotion. I wrote about it after posting my comment. Here's the chunk which answers your question: Where am I now? Heh. I mentioned that looking for duplicate files is harder than I thought it would be. I'm actually on my third try. The first one was when I thought "I can do this with a script", the second was with .NET, where I aimed bigger, but found not nearly big enough. So, I've just completed the work on the file crawler, and the next bit is submitting the crawl results to the index. I've done this part before, and I don't expect it to be particularly hard, but I have to find the time for it. After that, something resembling a UI (I am trying to solve a problem), then put the whole thing out there with a big fat "alpha" disclaimer (probably Apache license, since I'm using so much of their stuff). And that's what I'm doing, and where I'm at. Hope this helps. http://itgoon.blogspot.com/2009/08/wheres-that-file.html http://itgoon.blogspot.com/2009/08/wheres-that-file.html

mark_h 17y ago
Thanks very much; please consider posting here when you're done.
thaumaturgy 17y ago
Along the lines of what you're working on, I think what you really want is a form of network browser capable of locating files stored redundantly across a completely decentralized network. Done correctly, this would make notions like "backups" completely obsolete, and "my files" would exist on any computer that allowed me to log in to the appropriate service. Fast, decentralized, redundant network systems are a beast to engineer though, but it's do-able. Bonus points if you figure out how to let users store private data on such a network, and keep it private, while also allowing them to share specific files with specific individuals or groups of individuals. Pull that off, and you'll have the holy grail of data storage. I've only got some rough notes on this; I don't think I'd start trying to actually implement it for another two years yet. Beat me to it! I'd rather take a vacation. :-)

by itgoon 17y ago

mark_h 17y ago

Thanks very much; please consider posting here when you're done.

thaumaturgy 17y ago

Along the lines of what you're working on, I think what you really want is a form of network browser capable of locating files stored redundantly across a completely decentralized network. Done correctly, this would make notions like "backups" completely obsolete, and "my files" would exist on any computer that allowed me to log in to the appropriate service. Fast, decentralized, redundant network systems are a beast to engineer though, but it's do-able. Bonus points if you figure out how to let users store private data on such a network, and keep it private, while also allowing them to share specific files with specific individuals or groups of individuals. Pull that off, and you'll have the holy grail of data storage. I've only got some rough notes on this; I don't think I'd start trying to actually implement it for another two years yet. Beat me to it! I'd rather take a vacation. :-)