4 ms·
Ask HN: what problems in technical and scientific computing are worth solving?
TL;DR - if you work with high-performance computing what problems would you like to be solved in that area?
I work in finance and while there are things that demand a lot of expertise in high-performance computing, low-latency computing and big data and are technologically challenging and very interesting, I sometimes feel that there could be a better application for my skills.
My question is to HNers who work in fields that need a lot of computing horsepower: bioinfo, pharma, aerospace, geological survey, high energy physics, actually any kind of industry that does a lot of analytics.
What, in your opinion, are challenges in the area of high-performance computing, that - if solved - would make your life much easier: be it algorithms, infrastructure, tools, or workflows?
- ssylee 15y agoPaul Graham wraps it up pretty well in PyCon 2012 a few weeks ago: http://www.youtube.com/watch?v=R9ITLdmfdLI http://www.youtube.com/watch?v=R9ITLdmfdLI
- paulsutter 15y agoGenerally speaking, high performance computing challenges aren't valuable. And I say this as a guy whose last startup makes 50 million decisions a second and processes 10 petabytes a day (Quantcast). Making the technology scale was the -fun- part. The company gets paid for solving a specific business problem. A business problem that costs the world $600B a year (advertising). It's hard to start with a technical ability and find a startup opportunity. That is how we decided to start Quantcast - we had a spreadsheet of business ideas that might leverage our technical backgrounds. But it's a difficult path. It's easier to start with basic consumer or business needs and design a solution generally. The best business ideas you come up with are not likely to require the cool/fun high performance skills you (we) have. Murphy's law or something.
- strlen 15y ago> It's hard to start with a technical ability and find a startup opportunity. That is how we decided to start Quantcast - we had a spreadsheet of business ideas that might leverage our technical backgrounds. But it's a difficult path. Really interesting point. I am much in the same boat as you, I am interest in working on highly technical, mostly systems related challenges. However, I know that the goal of a company is to generate return for their shareholder. At the mean time, startup is a lot of hard, thankless, and grungy work (that makes tooling around with autoconf seem fun by comparison): unless you're genuinely passionate about the problem(s) you're solving, it isn't worth it. So until a deeply technical idea strikes me, I'll hold off on taking that jump.
- paulsutter 15y agoHere's one that came up in another thread: rewrite the core engine of MongoDB, yet keep all the existing API's and libraries. You could call it BongoDB. B for better. You can capture all their market share based mostly on technical prowess. Some of the problems are documented here: http://news.ycombinator.com/item?id=3837772 http://news.ycombinator.com/item?id=3837772
- bobak 15y agoHere is one example: http://www.infoq.com/news/2011/12/mongograph-qa http://www.infoq.com/news/2011/12/mongograph-qa