4 ms·
Andrew, this is an embarassingly parallel problem. The supercomputer hardware (interconnect) would be simply wasted. If you provide C++ source code that can
by dekhn 14y ago
Andrew,
this is an embarassingly parallel problem. The supercomputer hardware (interconnect) would be simply wasted. If you provide C++ source code that can be compiled with Native Client, and a list of tasks to run (binary command lines), Exacycle (http://googleresearch.blogspot.com/2012/12/millions-of-core-hours-awarded-to.html http://googleresearch.blogspot.com/2012/12/millions-of-core-...) is more appropriate. Based on our publicized results this wouldn't take very long (10 years on a desktop = less than an hour on Exacycle; you'd spend more time writing the program and the data analysis than the actual runtime).
however, I don't really see this problem as a high priority. pharma has bigger problems than to probe remote websites to find out what their competitors might be interested in.
- 0x0 14y agoWhy is Google insisting on the use of NativeClient? Not insinuating anything, but a NaCl-compiled distributed task would be an excellent fit for farming out to unsuspecting browser clients. (Reminds me about that sneaky javascript-based bitcoin miner that was injected on a few web sites some time ago)
- jey 14y agoBecause NaCl is a fully sandboxed way to run full-performance native code.
- dekhn 14y agoWe looked at several security and sandbox technologies, and NaCl was closest to our requirements: near-native performance, easy access to the NaCl software development team and supported C and C++. Strictly speaking, we use it as a "Native Server", rather than a "Native Client" (it runs on server-class machines, not desktop browsers). We've talked with Vijay Pande (of Folding@Home fame) about making "Folding@Chrome", but that would be opt-in (IE, you'd install a Chrome App). If you have an alternative sandbox and security framework, I'd be happy to hear about it.
- 0x0 14y agoThat's cool, I never really thought of using NaCl on the server side like that. So you are saying it is a better solution than any combination of Java, Ruby, SELinux, any other regular OS ACL, Xen, or any other virtualization platform?
- dekhn 14y agoAt least in my experience, most of the codes we expected to run were written in C or C++. We would like to support Java, but it's not an option right now (Native Client doesn't support Java, and I'd need to research the Java sandboxing technologies). Another issue is that we limit memory usage, and Java memory usage tends to be higher than C++ programs; ditto for virtualization. Note that virtualization isn't really a security solution on its own. I can't really see Ruby being the right approach for CPU-intensive computing (we're talking hundreds of thousands of cores, so it's worth investing in making the code efficient). BTW, I definitely think seccomp http://en.wikipedia.org/wiki/Seccomp http://en.wikipedia.org/wiki/Seccomp and linux containers http://lxc.sourceforge.net/ http://lxc.sourceforge.net/ or perhaps OpenVZ: http://wiki.openvz.org/Main_Page http://wiki.openvz.org/Main_Page could be engineered into a secure sandbox, without having to resort to virtualization (which I'd like to emphasize isn't really a security solution). This is called "container virtualization" (as compared to hardware/CPU-based), and has a number of very nice properties.
- drivebyacct2 14y agoYou might be interested in this: http://zerovm.org/ http://zerovm.org/ (No association but it seems highly relevant.)
- dalke 14y agoAbsolutely embarrassingly parallel. Thing is, I don't know what kinds of problems Xcelerate is interested in working on, I don't know what projects are given preference on Titan, and I don't even know if I'm in the right order of magnitude in my estimate. There are easier ways to make it more complex. For example, enumerate the first 15 atoms instead of the first 13. I believe this requires coordination between the enumeration nodes in order to reduce duplicate generation, but I don't know that subfield and labeled graph enumeration gets further away from hands-on chemical work. And stage 2 requires a combinatorial approach taking maybe 1,000x more time than the first stage. (That's a jazz hands level of waviness - I really don't know.) I once did work in molecular dynamics. In fact, I was one of the co-authors of NAMD. That's a rather intently explored field, and progress in it can feel rather incremental. What I proposed is one that no one has explored, to my knowledge. There's maybe 50 people in the world who would be able to do this work now, if they had time and hardware access (which they don't), and perhaps ~5,000 people who would be affected by knowing the result; mostly to know if they could do certain searches on the public web. So it's one with little competition and where the results would be immediately publishable. Not bad for a semester project, I think! I think it would be a good Master's thesis. And a lot different from the usual set of MD, docking/screening, folding projects that have consumed massive amounts of supercomputer time for the last three decades. Given that Xcelerate was talking about porting some code to the GPU, I suspect that that person already expect to "spend more time writing the program and the data analysis than the actual runtime." When I was doing MD work, I expected to spend about 6 months in simulation time and about 2 years in analysis for a PhD. The previous generation to me in the group had to build their own hardware before doing simulations, so I though that it's usually the case that the non-simulation time is longer than the simulation time. :) You said: "I don't really see this problem as a high priority. pharma has bigger problems than to probe remote websites to find out what their competitors might be interested in." So they should only work on the big problems that everyone else is doing? What about smaller problems which might help solve big problems? For example, 10 hours ago here on HN, mtgx posted a link titled "Patents Are Making Us Lose The Race Against Antibiotic-Resistant Bacteria" (to http://www.techdirt.com/articles/20130110/09590621628/world-economic-forum-warns-that-patents-are-making-us-lose-race-against-antibiotic-resistant-bacteria.shtml http://www.techdirt.com/articles/20130110/09590621628/world-... ), which reports that the "World Economic Forum's 8th Global Risks Report" suggests that "Rather than today's monopolistic hoarding [of data], what we need is more sharing of [pharmaceutical] knowledge." Let us suppose that they are correct, and we need to have more sharing of some knowledge between different companies, perhaps with "public or philanthropic funding to incentivize academic collaboration." One of the questions you could ask, if you had more information about what goes on inside of the companies, is: what's the overlap of the chemical space being tested by the different companies? Are they too similar? A recent paper on the topic - and one of the few such papers - is the recent "Big pharma screening collections: more of the same or unique libraries? The AstraZeneca–Bayer Pharma AG case". (See http://pipeline.corante.com/archives/2012/12/06/four_million_compounds_to_screen.php http://pipeline.corante.com/archives/2012/12/06/four_million... for a summary.) In it, they conclude that there is a "low overlap between both collections in terms of compound identity and similarity." They did it by using 2D ECFP4 fingerprints. A fingerprint is very much like a Bloom filter, where bits are set based on chemical features. It's one way to get information about identity and similarity. It's not strictly reversible, but it's leaky. For example, fragments can have certain characteristic patterns in the ECFP4 fingerprint, which can be used to infer some of the original structure. The InChI hashes are another way to test if two data sets contain identical structures. Two pharmas may want to compare them before during a pre-competitive collaboration, in order to determine the overlap between their two collections. What is their risk model? Can they outsource the evaluation knowing that no information can be leaked? Or is there an unexpectedly high leak rate which means that the hashes must never leave the internal network, and should be restricted to only a few trusted people? Right now, we don't know that answer. We believe that it reveals little information. I'm not so sure about that. If they don't reveal information, then certain new types of discussions can occur, which might (if the World Economic Forum is correct) help lead to the development of new drugs. Or, if it does reveal information, then it might help characterize how leaky that information actually is. Eg, it takes 1 month on an Exacycle machine to identify 1% of the structures with fewer than 15 atoms, and 10 years (estimated) to identify 1% of the structures with 17 atoms. That gives a usable risk model that doesn't currently exist.