5 ms·

AWS already has platforms for running and fine tuning OSS models that can run privately inside a VPC. If Azure and GCP don’t have equivalent capabilities alread

AWS already has platforms for running and fine tuning OSS models that can run privately inside a VPC. If Azure and GCP don’t have equivalent capabilities already, it is surely imminent. Seems pretty hard or impossible to beat cloud providers at their own game.

HeatrayEnjoyer 2y ago
If the hardware isn't in your physical possession you can't know that your data isn't being hoovered up. You can't end to end encrypt compute tasks (homomorphic processing is fiendishly uneconomical).
- TeMPOraL 2y ago
  True, but at this point we're leaving the realm of cryptography and theoretical infosec, and enter the realm of real-world security. In this realm, permissions are established by armies of lawyers across organizations and governments defining who can or cannot do things, and what happens when transgressions occur; here, "defense in depth" carries all the way to the threat of men with guns escorting you to jail. So it's true that you can't encrypt compute tasks of this type end-to-end, so you can't know if unauthorized parties mine your data. However, Microsoft is very unlikely to mine your data (for "you" being e.g. any of the many multinational corporations that already run all their office work through Azure-hosted Outlook, Office, SharePoint, etc.), or to let others mine it, because if it ever came out, your customers' lawyers would be after you, your lawyers would be after Microsoft, and the whole thing would explode into a multiple-billion-dollars shitshow and might even get a government or two involved. That's the working assumption that makes Microsoft well-positioned to eat any fledgling self-hosted DeepSeek market in the business space. They already have things set up at a level that is trusted by governments as well as corporations in critical industrial sectors, with huge financial and legal exposure. (Presumably Google and Amazon are in a similar position here, though I've only seen this personally with Microsoft/Azure, so that's what I can comment on.)
  - zie 2y ago
    > "defense in depth" carries all the way to the threat of men with guns escorting you to jail. For contract breach civil crime like this, there is zero chance it ends with jail time.
    - TeMPOraL 2y ago
      That's the typical case, true - but for many (most?) of the big multinationals, the worst case scenario for a hack involves people dying or some piece of critical infrastructure exploding. On top of that, "everything is securities fraud" - and since that does carry potential jail time, corporations generally try to avoid pissing off parties that would be able to frame a contract breach (and its consequences) in terms of investment fraud. EDIT: For starters, almost all data a multinational corporation generates and processes is subject to export control regulations, which are broad, full of special cases, vary over time, space and politics, and most importantly, violations of them come with huge fines and criminal penalties[0] for both businesses and individuals involved. The only reason Microsoft can get a corporation like this to migrate to O365 and run their back-office in Azure cloud is by solid, tested contractual guarantees that the data will be processed in ways that will keep the customer compliant with applicable regulations. Now, I'm not a lawyer, but it's not particularly hard to draw a line from "Microsoft snooping on enterprise customers" to securities fraud. I mean, even in context of hosting a DeepSeek derivative, we're talking about a cloud service offering enterprise customers secure training on company data. "Company data" may involve, e.g. detailed documentation or specs for software for designing advanced optical systems, which may sound benign until you make the connection[1]: "advanced optics" includes applications in advanced laser systems, which basically means weapons (e.g. ranging, missile targeting, anti-missile countermeasures). Obviously, regulators around the world (and the US in particular) would be very unhappy to see such information crossing through the wrong borders. For both the affected customers and the cloud service, this is high stakes game; a random startup isn't in a position to enter it. -- [0] - E.g. in US, up to $1M per violation and up to 20 years in prison, possibly at the same time; see https://www.bis.doc.gov/index.php/enforcement/oee/penalties https://www.bis.doc.gov/index.php/enforcement/oee/penalties. [1] - This was a real intro example used in export control training I went through some years ago.
- threeseed 2y ago
  > you can't know that your data isn't being hoovered up There is no evidence of this happening in the last 20 years. None. And if there was it would be the complete unravelling of the entire cloud concept. So you're talking about solving a problem no one has.
  - DrScientist 2y ago
    It's not sure much about hoovering, as targeted spying. Plenty of evidence of companies and governments using spying for commercial/national ( sometimes the same ) advantage. So let's say you are a big company, and suddenly the US government decides you are a competitor in a nationally strategic industry - is your data safe if held by a US company?
- kristjansson 2y ago
  If your data is too sensitive for AWS, you're in a different realm that most enterprise users.
  - coredog64 2y ago
    You have that right: https://aws.amazon.com/federal/secret-cloud/ https://aws.amazon.com/federal/secret-cloud/
guiambros 2y ago
> If Azure and GCP don’t have equivalent capabilities already, it is surely imminent GCP offers hundreds of models in its Vertex AI, including all "open source" (actually open weights) models, and the ability to fine tune for your specific needs. This blog post is from 2023 [1]. (disclaimer: I work at Google, but not on the Cloud team) [1] https://cloud.google.com/blog/products/ai-machine-learning/serve-open-source-llms-on-google-cloud https://cloud.google.com/blog/products/ai-machine-learning/s...

HeatrayEnjoyer 2y ago

If the hardware isn't in your physical possession you can't know that your data isn't being hoovered up. You can't end to end encrypt compute tasks (homomorphic processing is fiendishly uneconomical).

TeMPOraL 2y ago
True, but at this point we're leaving the realm of cryptography and theoretical infosec, and enter the realm of real-world security. In this realm, permissions are established by armies of lawyers across organizations and governments defining who can or cannot do things, and what happens when transgressions occur; here, "defense in depth" carries all the way to the threat of men with guns escorting you to jail. So it's true that you can't encrypt compute tasks of this type end-to-end, so you can't know if unauthorized parties mine your data. However, Microsoft is very unlikely to mine your data (for "you" being e.g. any of the many multinational corporations that already run all their office work through Azure-hosted Outlook, Office, SharePoint, etc.), or to let others mine it, because if it ever came out, your customers' lawyers would be after you, your lawyers would be after Microsoft, and the whole thing would explode into a multiple-billion-dollars shitshow and might even get a government or two involved. That's the working assumption that makes Microsoft well-positioned to eat any fledgling self-hosted DeepSeek market in the business space. They already have things set up at a level that is trusted by governments as well as corporations in critical industrial sectors, with huge financial and legal exposure. (Presumably Google and Amazon are in a similar position here, though I've only seen this personally with Microsoft/Azure, so that's what I can comment on.)
- zie 2y ago
  > "defense in depth" carries all the way to the threat of men with guns escorting you to jail. For contract breach civil crime like this, there is zero chance it ends with jail time.
  - TeMPOraL 2y ago
    That's the typical case, true - but for many (most?) of the big multinationals, the worst case scenario for a hack involves people dying or some piece of critical infrastructure exploding. On top of that, "everything is securities fraud" - and since that does carry potential jail time, corporations generally try to avoid pissing off parties that would be able to frame a contract breach (and its consequences) in terms of investment fraud. EDIT: For starters, almost all data a multinational corporation generates and processes is subject to export control regulations, which are broad, full of special cases, vary over time, space and politics, and most importantly, violations of them come with huge fines and criminal penalties[0] for both businesses and individuals involved. The only reason Microsoft can get a corporation like this to migrate to O365 and run their back-office in Azure cloud is by solid, tested contractual guarantees that the data will be processed in ways that will keep the customer compliant with applicable regulations. Now, I'm not a lawyer, but it's not particularly hard to draw a line from "Microsoft snooping on enterprise customers" to securities fraud. I mean, even in context of hosting a DeepSeek derivative, we're talking about a cloud service offering enterprise customers secure training on company data. "Company data" may involve, e.g. detailed documentation or specs for software for designing advanced optical systems, which may sound benign until you make the connection[1]: "advanced optics" includes applications in advanced laser systems, which basically means weapons (e.g. ranging, missile targeting, anti-missile countermeasures). Obviously, regulators around the world (and the US in particular) would be very unhappy to see such information crossing through the wrong borders. For both the affected customers and the cloud service, this is high stakes game; a random startup isn't in a position to enter it. -- [0] - E.g. in US, up to $1M per violation and up to 20 years in prison, possibly at the same time; see https://www.bis.doc.gov/index.php/enforcement/oee/penalties https://www.bis.doc.gov/index.php/enforcement/oee/penalties. [1] - This was a real intro example used in export control training I went through some years ago.
threeseed 2y ago
> you can't know that your data isn't being hoovered up There is no evidence of this happening in the last 20 years. None. And if there was it would be the complete unravelling of the entire cloud concept. So you're talking about solving a problem no one has.
- DrScientist 2y ago
  It's not sure much about hoovering, as targeted spying. Plenty of evidence of companies and governments using spying for commercial/national ( sometimes the same ) advantage. So let's say you are a big company, and suddenly the US government decides you are a competitor in a nationally strategic industry - is your data safe if held by a US company?
kristjansson 2y ago
If your data is too sensitive for AWS, you're in a different realm that most enterprise users.
- coredog64 2y ago
  You have that right: https://aws.amazon.com/federal/secret-cloud/ https://aws.amazon.com/federal/secret-cloud/

guiambros 2y ago

> If Azure and GCP don’t have equivalent capabilities already, it is surely imminent GCP offers hundreds of models in its Vertex AI, including all "open source" (actually open weights) models, and the ability to fine tune for your specific needs. This blog post is from 2023 [1]. (disclaimer: I work at Google, but not on the Cloud team) [1] https://cloud.google.com/blog/products/ai-machine-learning/serve-open-source-llms-on-google-cloud https://cloud.google.com/blog/products/ai-machine-learning/s...