9 ms·
What we really need is a model that you can run on your own hardware on site. I could never use this for business because they're reading everything you send th
by MagicMoonlight 4y ago
What we really need is a model that you can run on your own hardware on site. I could never use this for business because they're reading everything you send through it, but let me run it on my own server and it would be unbelievably useful.
Imagine being able to ask your workplace server if it has noticed any unusual traffic, or to write a report on sales with nice graphs. It would be so useful.
- jahewson 4y ago“on site”? Medical records are in the cloud already.
- szundi 4y agoWhat is “the cloud” - that’s the question
- gffrd 4y agoAs taken from the cover page of the July, 2018 edition of AARP Weekly.
- MisterBastahrd 4y agoYes. But they aren't being shared with third party AIs. Sharing personal medical information with OpenAI is a good way to get both your medical org to get ground into dust under a massive class action lawsuit, not to mention huge fines from the government.
- sebzim4500 4y agoWhy do you think sharing the data with OpenAI is legally any different than storing it on AWS/Azure/GCP/Whatever else they are using?
- unusualmonkey 4y agoGCP respects hipaa (google 'gcp hipaa baa'). Does OpenAPI?
- sebzim4500 4y agoIf they don't now they will in the future, if they think there is money to be made. Why wouldn't they? They could even charge a premium for the service.
- dragonwriter 4y agoGCP/AWS/Azure have HIPAA programs in places, and will, consequently, sign HIPAA BAAs to legally perform as Business Associates of covered entities, fully responsible for handling PHI in accord with HIPAA rules (for certain of their services.) OpenAI itself does not seem to offer this for either its UI or API offerings. Microsoft, OTOH, does now offer a HIPAA BAA for its Azure OpenAI service, which includes ChatGPT (which means either they have a bespoke BAA with OpenAI that OpenAI doesn’t publicly offer, or they just are hosting their own ChatGPT instance, a privilege granted based on them being OpenAI’s main sponsor.)
- geysersam 4y agoThat's ridiculous. Sure if you put it into ChatGPT today that's a problem. But if you have a deal with the company providing this service, and they are certified to follow the relevant regulations around sensitive data, why would that be different from any other cloud service? If this proves actually useful I guess such agreements could be arranged quite quickly.
- porknubbins 4y agoYes, almost all eDiscovery is managed by cloud vendors as is, and no one worries about waiver of privilege to these companies. The only concerns I’ve heard have been relates to foreign companies or governments not wanting their data to be hosted in a foreign country. But domestically it should be fine to have a chatgpt legal where data is discarded not saved.
- chaxor 4y agoIt's only been a few hours since Ring was hacked... a system run by a large company which assured everyone they were taking good care of their data. Surely the wonderful Amazon, with all of it's massive capital, could do the simple thing of encrypting incredibly sensitive and private user data? Right?
- peterashford 4y agoYes, but their access is strictly controlled. There's a lot of regulation about this stuff
- geysersam 4y agoIf the chatbot technology proves useful I'm sure OAI could make some agreement to not store sensitive data.
- peterashford 4y agoyes - you could add regulation
- serf 4y agoright, but 'the cloud' isn't a singular monolithic database that everyone inputs data into for a result. most of the AI offerings on the table right now aren't too dissimilar from that idea in principle.
- Semioj 4y agoThat's not entirely true. Google has a contract with the biggest hospital operator in the USA. Tx also to some certificate they aquires
- chaxor 4y agoIs and ought https://en.m.wikipedia.org/wiki/Is%E2%80%93ought_problem https://en.m.wikipedia.org/wiki/Is%E2%80%93ought_problem
- bradleyjg 4y agoI don’t think on site is going to be necessary. Even the US intelligence community trusts that Amazon isn’t spying on the spies. But a model that can run on a private cluster is certainly something that there’s going to be demand for. And once that exists there’s no reason it couldn’t be run on site. You can see why OpenAI doesn’t want to do it though. SaaS is more lucrative.
- lillecarl 4y agoVMware charges people per GB RAM attached to a VM. Selling on-prem software on consumption is very much possible. It's closed source software, so as long as they require 443 outbound to tick consumption that'd work.
- jsjohnst 4y ago> You can see why OpenAI doesn’t want to do it though. Except they already do offer private cluster solutions, you just need usage in the hundreds of millions of tokens per day before they want to talk to you (as in they might before that, but that’s the bar they say on the contact us page).
- hailwren 4y ago> Even the US intelligence community trusts that Amazon isn’t spying on the spies I’m not sure what you mean by this, but it’s incorrect. Sensitive USG information is not processed on Amazon’s commercial offering. > The Amazon-built cloud will operate behind the IC’s firewall, or more simply: It’s a public cloud built on private premises. [1] I think this is what you’re referring to. 1 - https://www.theatlantic.com/technology/archive/2014/07/the-details-about-the-cias-deal-with-amazon/374632/ https://www.theatlantic.com/technology/archive/2014/07/the-d...
- jimhi 4y agoThey are referring to this https://aws.amazon.com/govcloud-us/?whats-new-ess.sort-by=item.additionalFields.postDateTime&whats-new-ess.sort-order=desc https://aws.amazon.com/govcloud-us/?whats-new-ess.sort-by=it...
- make3 4y agothe problem is that if you steal the weights then you can serve your own gpt4, and it's very hard to prove that what you're serving is actually gpt4. (or you could just start using it without paying ofc)
- RealityVoid 4y agoPresumably, if you give it identical prompts you get identical answers?
- Sander_Marechal 4y agoNo, these NLPs aren't idempotent. Even if you ask ChatGPT the same question multiple times you will get different answers.
- LawTalkingGuy 4y agoThat's the feature of chat - it remembers what has been said and that changes the context in which it says new things. If you use the API it starts fresh each time, and if you turn down the 'temperature' it produces very similar and identical answers.
- outside1234 4y agoyes - they are multinomial distributions over answers essentially
- parentheses 4y agoThis may be an implementation detail to obfuscate GPT weights. OR it was to encourage selecting the best answers to further train the model.
- textninja 4y agoPseudo random numbers are injected into the models via its temperature settings, but OpenAI could seed that to get the same answers with the same input. I’m going out on a limb here with pure speculation but given the model, a temperature, and a known text prompt, OpenAI could probably reverse engineer a seed and prove that the weights are the same.
- slt2021 4y agomaybe we implement tokenizer+first layer in Javascript on client side and that is enough to preserve raw data on client side and send to GPT only first layer (which is a vector of float values anyway) matrix gets decoded into text on the client side in Javascript, so we receive send and receive from chatGPT only vector of floats (obfuscation?)
- geysersam 4y agoIt's a good idea but it seems quite easy to invert the first layer mapping. And the output of the last layer you can easily steal just by doing whatever would've been done in the client.
- nickpeterson 4y agoCould open ai just offer letting you upload a key and use it for interaction with the model? Basically encrypt the model with the key and all the request and responses are all secure? I’m probably oversimplifying but it feels doable.
- slt2021 4y agothe goal is how to use chatGPT without sending plain text to OpenAI (to preserve privacy, make sure openai is unable to even see plain customer data)
- andai 4y agoMaybe if we could speak with GPT-4 instead of OpenAI ;)
- abudabi123 4y agoWill the nonpareil paraquet make original discoveries and inventions from protein folding and stem cells results, GPT-X interfacing with DeepMind?
- colinsane 4y ago> What we really need is a model that you can run on your own hardware on site. we won’t have that until we come up with a better way to fund these things. “””Open””” AI was founded on that idea, had the most likely chance of anyone in reaching it: even going into things with that intent they failed and switched to lock down the distribution of their models, somehow managed to be bought by MS despite the original non-profit-like structure. you just won’t see what you’re asking for for however long this field is dominated by the profit motive.
- sounds 4y agoNah, it's already being done for GPT-3's competitors and will likely be done soon for GPT-4's competitors https://arstechnica.com/information-technology/2023/03/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi/ https://arstechnica.com/information-technology/2023/03/you-c...
- systemvoltage 4y agoCurious why even companies at the very edge of innovation are unable to build moats? I know nothing about AI, but when DALLE was released, I was under the impression that the leap of tech here is so crazy that no one is going to beat OpenAI at it. We have a bunch now: Stable Diffusion, MidJourney, lots of parallel projects that are similar. Is it because OpenAI was sharing their secret sauce? Or is it that the sauce isn’t that special?
- elevaet 4y agoI think it's because everyone's swimming in the same bath. People move around between companies, things are whispered, papers are published, techniques are mentioned and details filled in, products are backwards-engineered. Progress is incremental.
- PaulHoule 4y agoGoogle got a patent on transfomers but didn't enforce it. If it wasn't for patents you'd never get a moat from technology. Google, Facebook, Apple and all have a moat because of two sided markets: advertisers go where the audience is, app makers go where the users are. (There's another kind of "tech" company that is wrongly lumped in with the others, this is an overcapitalized company that looks like it has a moat because it is overcapitalized and able to lose money to win market share. This includes Amazon, Uber and Netflix.)
- javier2 4y agoYou mean hallucinated graphs and word prediction unusual traffic? No, I get that the models are very impressive, but im not sure they actually reason
- drdeca 4y agoWhat do you mean by “actually reason”? And, presumably you wouldn’t have the model generate the graph directly, but instead have it generate code which generates the graph. I’m not sure what they had in mind for the “unusual traffic” bit.
- ElFitz 4y agoFor that I'd suggest using Langchain with Wolfram Alpha. It's already been done and discussed: - https://news.ycombinator.com/item?id=34422122 https://news.ycombinator.com/item?id=34422122 - https://news.ycombinator.com/item?id=34422627 https://news.ycombinator.com/item?id=34422627
- vintermann 4y agoThe thinking elevator So the makers proudly say Will optimize its program In an almost human way. And truly, the resemblance Is uncomfortably strong: It isn't merely thinking, It is even thinking wrong. Piet Hein wrote that in reference to the first operator-free elevators, some 70+ years ago. What you call hallucination, I call misremembering. Humans do it too. The LLM failure modes are very similar to human failure modes, including making up stuff, being tricked to do something they shouldn't, and even getting mad at their interlocutors. Indeed, they're not merely thinking, they're even thinking wrong.
- mrtranscendence 4y agoI don't think it's very salient that LLMs make stuff up, or can be manipulated into saying something they have been trained not to say. An LLM applies a statistical model to the problem of probability assignment over a range of tokens; a token of high probability is selected and the process repeats. This is not what humans do when humans think. Given that GPT-4 is a simply large collection of numbers that combine with their inputs via arithmetic manipulation, resulting in a sequence of numbers, I find it hard to understand how they're "thinking".
- zmmmmm 4y ago> What we really need is a model that you can run on your own hardware on site So, LLaMA? It's no chat gpt but it can potentially serve this purpose
- la64710 4y agoJust ask OpenAI and it will build it :)
- f0e4c2f7 4y agohttps://github.com/tatsu-lab/stanford_alpaca https://github.com/tatsu-lab/stanford_alpaca Tada! Literally runs on a raspberry pi (very slowly). GPT models are incredible but the future is somehow even more amazing than that. I suspect this will be the approach for legal / medical uses (if regulation allows).
- qualudeheart 4y agoThat model will be out in a few years. GPT-3 175b only took two years until someone trained an open source equivalent that could run on a few gpu devices.
- ElFitz 4y agoOr using homomorphic encryption. I remember some managing to run inference on encrypted images. See - https://www.zama.ai/post/encrypted-image-filtering-using-homomorphic-encryption https://www.zama.ai/post/encrypted-image-filtering-using-hom... - https://news.ycombinator.com/item?id=31933995 https://news.ycombinator.com/item?id=31933995 - https://news.ycombinator.com/item?id=34080882 https://news.ycombinator.com/item?id=34080882 - https://news.ycombinator.com/item?id=25786154 https://news.ycombinator.com/item?id=25786154 - https://news.ycombinator.com/item?id=30626182 https://news.ycombinator.com/item?id=30626182
- runnerup 4y agoHomomorphic encryption has a 1,000,000x performance disadvantage. So maybe in 30 years as we approach the Landauer limit, but not in our generation.
- insanitybit 4y ago> So maybe in 30 years as we approach the Landauer limit, but not in our generation. I feel like 30 years is squarely within our generation
- tga_d 4y agoDepends on the definition of "generation" being used. One definition of generation is "about 30 years", i.e., the amount of time it takes to go from infancy to raising a child. See definition 6 (as of time of writing): https://en.wiktionary.org/wiki/generation#Noun https://en.wiktionary.org/wiki/generation#Noun
- insanitybit 4y agoHuh, thanks. I would not have guessed.
- stephenkingsley 4y ago[dead]
- ludovicianul 4y agoIsn't Azure OpenAI suppose to do this? (not locally, but private)
- cutler 4y agoThis is Microsoft we're talking about. Hail the new old overlord.
- PaulHoule 4y agoModels you can run locally are coming soon.