6 ms·
Halfway down the article contains some great charts with comparisons to other relevant models, like Mistral-7B for the on-device models, and both gpt-3.5 and 4
by cube2222 2y ago
Halfway down the article contains some great charts with comparisons to other relevant models, like Mistral-7B for the on-device models, and both gpt-3.5 and 4 for the server-side models.
They include data about the ratio of which outputs human graders preferred (for server side it’s better than 3.5, worse than 4).
BUT, the interesting chart to me is „Human Evaluation of Output Harmfulness” which is much, much ”better„ than the other models. Both on-device and server-side.
I wonder if that’s part of wanting to have gpt as the „level 3”. Making their own models much more cautious, and using OpenAI’s models in a way that makes it clear „it was ChatGPT that said this, not us”.
Instruction following accuracy seems to be really good as well.
- crooked-v 2y agoI want to know what they consider "harmful". Is it going to refuse to operate for sex workers, murder mystery writers, or people who use knives?
- its_ethan 2y agoThe caption for the image gives a little more insight into "harmful" and one of the things it mentions is factuality - which is interesting, but doesn't reveal a whole lot unless they were to break it out by "type of harmful".
- hotdogscout 2y agoI bet it's the usual double standards the AI one percenters cater to. No sex because apparently it's harmful yet never explained why. No homophobia/transphobia if you're Christian but if you're Muslim it's fine.
- arthur_sav 2y agoThey'll inject whatever ideology / dogma is "the current thing" into this.
- HeatrayEnjoyer 2y ago[flagged]
- mvandermeulen 2y agoDo they have exclusive rights?
- soygem 2y agoYes, what's your point?
- HeatrayEnjoyer 2y agoRepeating right wing rhetoric is corrosive and a red flag that the user is at best repeating it out of ignorance, but likely knows very well what they are doing. We don't go pouring HCL on things we care about, nor do we accept the presence of those who do.
- rvnx 2y ago"Think Different"
- m463 2y agoBet it depends on the country. In the USA, you won't be able to ask about sex, but you can probably ask about tank man.
- jeroenhd 2y agoI would've thought the same until Microsoft started hiding tank man results in Bing. I'm not so sure if companies will start training different models for every oppressive regime.
- t-writescode 2y agoOh? https://www.bing.com/images/search?q=tank%20man https://www.bing.com/images/search?q=tank%20man Looks visible, to me. Tiananmen Square even shows Tank Man on the first page, 13th and 15th entry, for me. Admittedly, I expected it more quickly on Tiananmen Square; but that might be because I, as a person, forgot that it's also a literal square with more stuff going on at it than a single moment in history.
- jan3024 2y ago[dead]
- lelandfe 2y ago2021, a classic, “we made a whoopsie” response. https://theguardian.com/technology/2021/jun/04/microsoft-bing-tiananmen-tank-man-results https://theguardian.com/technology/2021/jun/04/microsoft-bin...
- Aerbil313 2y agoNone of the use cases they presented in WWDC using Apple Intelligence was creative writing. There is one, that uses ChatGPT explicitly: > And with Compose in Writing Tools, you can create and illustrate original content from scratch. https://www.apple.com/apple-intelligence/ https://www.apple.com/apple-intelligence/
- tonynator 2y agoSo it's not going to be better than other models, but it will be more censored. I guess that might be a selling point for their customer base?
- dghlsakjg 2y agoiPhone share is ~59% of smartphones in the US. Their customer base is effectively all demographics.
- tonynator 2y agoThose who dislike censorship and enjoy hacking avoid iPhones for obvious reasons.
- duxup 2y agoI feel that way, have an iPhone.
- ihumanable 2y agoYea, same. I have literally no desire to hack and fuck around with my personal cell phone, doing so would take away time from the hacking I actually want to do.
- tonynator 2y agoOne option you can't hack on, the other you can but don't have to at all. Why choose the former? Are you so sure you'll never want to do anything on your phone that's unapproved by your corporate overlords?
- overstay8930 2y agoPeople who understand cybersecurity hygiene use iPhones for obvious reasons
- 2y ago
- causal 2y agoRefusing to answer any question would result in a perfect score for the first chart since it says nothing of specificity