7 ms·
The gap that firms are trying to bridge is _huge_. I work in finance (slow finance: insurance), but finance technical enough to have our own PhD-level quants bu
by wjnc 11y ago
The gap that firms are trying to bridge is _huge_. I work in finance (slow finance: insurance), but finance technical enough to have our own PhD-level quants building code. They are running simulations taking hours, so scaling up on # cores would seem the first step to take. But it's not that the quants don't know parallelization, it's the IT-architecture not allowing to use AWS at all. And well, many cores on your own premises is hugely expensive because they are idle most of the time. And as a sidestep, most quants are not the first you think of to woo architectures' support. (This is probably easier for firms that trade for own profit.)
Point being: improvements in AI / machine learning and other statistical techniques are huge, and analytical demand exists, but 'old corporations' need cultural change before they can start working with and embedding startup-like technology in their workflows.
- mcguire 11y agoThis is pretty common outside finance as well. "We can't let our data touch any one else's network!" And, of course, we can't bother to get our IT group on the smarts wagon, so we're massively inefficient all the way around.
- Someone1234 11y agoI'm playing devil's advocate a little: There are sometimes legitimate legal hurdles to having your data hit external networks. For example, if you're a medical insurance company, then you have to get everything HIPAA legal.
- sologoub 11y agoLooks like AWS is HIPAA complaint at least for some services: "Customers may use any AWS service in an account designated as a HIPAA account, but they should only process, store and transmit PHI in the HIPAA-eligible services defined in the BAA. There are six HIPAA-eligible services today, including Amazon EC2, Amazon EBS, Amazon S3, Amazon Redshift, Amazon Glacier, and Amazon Elastic Load Balancer." http://aws.amazon.com/compliance/hipaa-compliance/ http://aws.amazon.com/compliance/hipaa-compliance/ Notably, I don't see Elastic MapReduce. It seems that if you wanted to run heavy data crunching, you'd have to run you own cluster on EC2.
- jtbigwoo 11y agoI've been on projects where we used Rackspace cloud servers or Amazon EC2 for HIPAA-compliant insurance enrollment systems. There was extra paperwork and we did a bunch of security testing, but it wasn't too bad. In both cases the projects were controlled by marketing departments who were much more concerned with cost per customer than with maintaining total control over every aspect of the system.
- mcguire 11y agoSure, sure, and some of our data is even sensitive in that way (cough, HR, cough). But I'm sure Amazon or someone else can handle that kind of requirement. We can't even open the discussion, though.
- the8472 11y ago> "We can't let our data touch any one else's network!" In some industries that can be necessary if you want to stick to the letter of the law, especially here in europe. You can outsource the processing, but then your provider has to meet fairly stringent certifications, you have the necessary "submitting to 3rd parties for purpose-specific processing" disclaimers in your privacy policies and a signed data processing agreement covering personal data compatible with national laws, which vary between countries. The microsoft ireland email case[1] also made some data protection ombudsmen suggest that the whole safe harbor deal with US companies should be considered null and void because they cannot effectively guarantee the safety of your data, even more so given the threat of national security letters + gag orders. And without that deal you can't use their cloud solutions for sensitive data. Personally identifying data has pretty high hurdles. Medical data (which might be processed by insurances?) raises even higher bars.
- sseveran 11y agoIn fact AWS will sign a HIPAA business agreement for medical data (http://aws.amazon.com/compliance/hipaa-compliance/ http://aws.amazon.com/compliance/hipaa-compliance/).
- kejaed 11y agoThey'll even help you with ITAR controls too: http://aws.amazon.com/govcloud-us/faqs/ http://aws.amazon.com/govcloud-us/faqs/
- superuser2 11y agoWhere is the line? Everyone seems to think "cloud" services threaten user privacy, but "cloud" is a marketing term. What about traditional VPSes? Managed dedicated servers? Ordinary dedicated servers? Leased servers collocated in a 3rd party datacenter? Owned servers collocated in a 3rd party datacenter? What if you use their "remote hands" service? Is there a meaningful difference between your leased office floor and your leased datacenter space? I would argue not. Do you have to own the land? What about leased vs. owned servers if they're in your office? Even if you have outright owned servers on outright owned land (unlikely b/c mortgage, but whatever), how are they managed? In many small businesses (including medical practices), they're managed by a "small business IT consulting" company which has both remote and regular on-premise access. You're trusting them to the same extent you would be trusting AWS. Is there a meaningful difference? What about the proprietary software you run? A large Electronic Medical Records package which probably contains extremely sensitive data about you is technically deployed onsite, but deployment and administration are performed remotely by the vendor's team in India. (I worked for a small biz IT consulting firm that supported the underlying hardware/Windows for one of these installations.) What about the Windows and other proprietary software with internet access that's literally everywhere? You may think you've achieved a morally superior position of trusting no one but yourself when you forego The Cloud, but I argue that's generally not the case except in the most extremely self-reliant cases (no vendor support, no contractors, no proprietary software). Those cases are not at all representative of the average business which refuses to go AWS.
- vosper 11y agoYeah. The company I work for was acquired by a big player in the industry, and one of the first things they wanted to do was move us off AWS, because the COO sees Amazon as a competitor and thinks that they'll go snooping in our databases to get a competitive advantage [1]. We managed to stave it off by coming up with a vague plan for how we'd move critical infrastructure to hardware, then shelving it indefinitely. AWS was (and is) critical to our velocity - unlike every other team in the company, we don't wait 6 weeks to start working on something while IT eventually gets around to writing a hardware purchase order and racking the gear. We don't compete for "the staging environment" because we can have as many environments as we want. [1] This is a ludicrous idea. My two main rebuttals are: - AWS takes in more revenue than our company, and they would destroy that business if it ever got out that their engineers were snooping in customer data - Netflix is a much more direct competitor than we are (they compete with Amazon Video) and also a much heavier user of (and cheerleader for, even) AWS
- nimblegorilla 11y agoIt might be time for tinfoil hats, but maybe Amazon only competes with Netflix because Netflix demonstrated how awesome Amazon's servers work for video delivery?
- bashinator 11y agoNetflix doesn't use AWS for the actual video delivery.
- throwawayaway 11y agooh no, never engineers. always maintain a layer of plausible deniability, those engineers might turn coat and rat the organisation out you see.
- hinkley 11y agoI really love it when I have to head up a project that costs the company $100-250,000 in developer time and the only reason we have to do it is that we can't get the business to buy us another $10,000 in hardware. Actually I lied. What I really love is when one customer threatens to walk if they have to buy a $3000 machine to run our new version, and the whole team spends 1-2 months doing nothing but trying to keep that account. When we could just gift them a server and save ourselves a mountain of grief and a ton of development resources for, you know, landing new accounts. If someone could start teaching basic math in business school, that would make me so happy.
- raverbashing 11y agoOf course, why have them pay a couple thousand dollars for an AWS installation when they can buy a "corporate solution" for "3 zeroes more"
- saganus 11y agoOne totally uninformed question: Do you think this could be solved with stacks of Parallela boards? Just curious, because that's kind of like the target market for the boards right? Cheap , multi-core processing?
- wjnc 11y agoTechnically: sure. But I think you are touching some different skill sets going on a continuum from (1) 'sequential simulations on regular machine' to (2) 'parallel simulation on very high end, regular machine', all the way to (3) 'massively parallel on non-regular machine'. From (1) to (2) you can manage with existing staff and limited time. From (2) to (3) is quite the change in mindset, training et cetera.
- saganus 11y agoI guess that makes sense. It does seem like a more exotic solution. Especially considering the amount of different skills that whoever is building such system would need to have. Maybe in some years we'll see if the Parallela paradigm takes off and then maybe there will be more quants out there capable of using such boards.
- sbenario 11y agoThat's one of the problems we're trying to solve with Ufora. Because it's so much more difficult to write "high performance parallel simulations", we want to make it so you can write whatever code you want, and let the platform figure out how to make it fast, parallel, and distributed.
- jacquesm 11y agoI'd look at checking if a GPU is a good match for the problem before thinking of even more exotic solutions.
- virmundi 11y agoSo could you take a page from Pixar or Seti and use company laptops while they're idle? Even if it was just on the quants laptops, it could help out. There would be no PO for the hardware. Shouldn't take to long to set up a simple system, but that could be the over optimistic American in me.
- jdboyd 11y agoWhere does Pixar say they use company laptops while they are idle?
- mryan 11y agoI've not read about Pixar doing this, but I found it interesting that IKEA does: http://www.cgsociety.org/index.php/CGSFeatures/CGSFeatureSpecial/building_3d_with_ikea http://www.cgsociety.org/index.php/CGSFeatures/CGSFeatureSpe...
- virmundi 11y agoYea, I can't find Pixar doing this. I thought I remembered an article from years ago. Other companies did it as well. https://books.google.com/books?id=QxAEAAAAMBAJ&lpg=PA51&ots=MJAXAN7zms&dq=rendering%20movie%20idle%20pcs&pg=PA51#v=onepage&q=rendering%20movie%20idle%20pcs&f=false https://books.google.com/books?id=QxAEAAAAMBAJ&lpg=PA51&ots=... But they might since RenderMan supports it, http://renderman.pixar.com/resources/current/tractor/tractor_nimby.html http://renderman.pixar.com/resources/current/tractor/tractor...
- raverbashing 11y ago> Yea, I can't find Pixar doing this Maybe because it doesn't make sense to copy a lot of model/texture data to/from a slow laptop for processing.
- njharman 11y agoThe data is small compared to processing required. Laptops aren't that slow. They are as fast as the fastest "servers" from a few/several years ago. Finally, most employees proly don't have laptops, OP misspoke.
- tdees40 11y agoConfirmed. I'm in insurance too - variable annuities specifically - and it's the most perfect problem for parallelization. In practice everyone maintains their own in-house grid (which is massively expensive and idle 90% of the time), and the technical staff lacks any serious chops to tackle GPUs. In practice the only way we've found to solve this is to go around corporate IT, so we do things like buying GPU cards and jamming them into boxes ourselves.
- ethbro 11y agoAs someone related to the field, I see a fundamental lack of quality in in-house IT when faced with problems of this magnitude. Which isn't unreasonable. IT evolved to be able to provision systems and roll out approved software. Maybe there's a development team or two to develop in-house extensions or software (though always viewed as a cost center). Therefore, the department the organization's rules are set up to funnel you through is simultaneously incapable of fulfilling this need. In practice, this leads to a ton of business specialty + devOps types from anyone who can hack Python in order to Get Stuff Done. (For those of you that don't deal with 1,000+ headcount organizations, be thankful ;) )
- tdees40 11y agoI think the big thing at the finance companies I've been at is that corporate IT was created to enable people to use Microsoft Office applications. There is no real provision for how to deal with developers writing code, and forget about hardware.
- ethbro 11y agoExactly, I think the telling question to any corporate IT team (not the head of all corporate IT) should be "When did you last identify, trial, and then make available to users a software package / process / cloud service without first receiving a request from management that you do so?" In my experience, the answer is usually never. Corporate IT exists to fulfill requirements from above, not to explore how things can be done better.
- 11y ago
- GFK_of_xmaspast 11y agoIf most of your cores are idle most of the time, you need to be doing things like searching bigger state spaces.
- mseebach 11y agoIT shops are inherently conservative, that's their job. The trick is to get the higher-ups interested - they have the power to redirect IT, and are inherently (if laggingly) interested in making employees as effective as possible. Funnily enough, a fluff-piece in Bloomberg is a pretty good way of achieving it. Tomorrow, some manager will pop into a quant's office with this article in his hand and ask "Are you doing this? Why not?" and that's how the ball gets rolling on opening up. Once upon a time, getting remote access to email was a "this will never happen", then an executive saw someone with a BlackBerry at the golf club. Once upon a time, getting email access on anything other than a BlackBerry was a "this will never happen", then an executive saw someone with an iPhone.
- Florin_Andrei 11y ago> IT-architecture not allowing to use AWS at all IPSec all traffic, including internal traffic between instances? Encrypt all volumes with keys not written anywhere within the cloud? That should do it, no?
- mariusandreiana 11y agoUnrelated: Salut Florin, cred ca ne stim de pe RLUG. Veneam pe la tine in Drumul Taberei sa facem chestii pe linux. Daca vrei sa mai keep in touch, am adresa in profil. Numai bine!