9 ms·

Meet “Claude”: Anthropic’s rival to ChatGPT

188 points by goodside 4y ago

nathias 4y ago
we need less censorious AIs not more ... the claim that it's somehow 'ethical' to have a guy baking in his opinions about things in a tool used globally is absurd to anyone who ever read anything about ethics
- Al-Khwarizmi 4y ago
  My hope is on a non-American alternative. The American society seems too engulfed by puritanism to produce a less straightjacketed chatbot.
  - unnouinceput 4y ago
    Watch the show Person of Interest (https://www.imdb.com/title/tt1839578/ https://www.imdb.com/title/tt1839578/). Somewhere around middle of season 2 it's explained why a self-aware AI is straightjacketed. Also the series shows what happens when one is not.
    - flanked-evergl 4y ago
      I thought that is a fictional show.
- astrange 4y ago
  You're welcome to try GPT3 before instruction tuning. It doesn't work at all. "Uncensored" AIs especially don't work for women because they'll immediately start writing erotica.
  - nathias 4y ago
    I don't have an issue with instruction tuning, but it's disengenious to pretend the biases inherent in the instruction tuning are a good thing and 'ethics'.
    - astrange 4y ago
      It's always got biases (it's made of them) so some biases must be better than others. https://spetharrific.tumblr.com/post/26600309788/sussman-attains-enlightenment/amp https://spetharrific.tumblr.com/post/26600309788/sussman-att... The untuned model isn't "an average of all opinions on Earth" or anything either.
  - topynate 4y ago
    GPT3 works quite well at following instructions if adequately prompted, just not as well as ChatGPT. ChatGPT was trained separately for ability to follow instructions and for harmlessness (sic). Not only is a non-moralising ChatGPT possible, one was actually created during the research process. You may also wish to know that most readers of erotica are women.
    - astrange 4y ago
      > You may also wish to know that most readers of erotica are women. I know that, but that doesn't mean they want to get it anytime they prompt with their names. There's actually multiple anecdotes from OpenAI employees about this happening to them (with prompts like "write a diary entry about my day").
      - krisoft 4y ago
        > with prompts like "write a diary entry about my day") Although I don't know what sensible output anyone would hope from that prompt.
        astrange 4y ago
        Don't have a diary on me, but seems to me if you fed it some examples it'd work fine on ChatGPT.
- viraptor 4y ago
  We already had a learning AI without hard limits. Remember Microsoft's Tay? Ah, right, it was shut within days because it quickly became an asshole. (https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...) It's not even going to be objectively wrong a lot of the time. For example slaves are indeed a pretty efficient way to run a business. It's our limits and ethics that stops (most of) us from doing it. Without ethics, you'll likely always end up with a 4chan-bot instead of whatever you intended.
  - flanked-evergl 4y ago
    > Remember Microsoft's Tay? Ah, right, it was shut within days because it quickly became an asshole. If we consider the context, which is not something that posts on twitter under Microsoft's brand name, but something you communicate with in private. Who exactly are you worried about here, the person who will prospectively coerce the language model into being an asshole? If they don't want to do that, they could just not do that. If I make ChatGPT say something egregious, and post that on Twitter or Facebook, I'm posting it, and I'm liable, just as I would be liable if I used a word processor with spell checking to make text and post it on Twitter or Facebook. > Without ethics, you'll likely always end up with a 4chan-bot instead of whatever you intended. If I ask it to explain quantum physics to me in the style of Donald Trump because it is funny, and it does it (as ChatGPT used to do), who exactly is being harmed and under what system of ethics, because as you may know, ethics is not objective or universal.
    - _flux 4y ago
      > If I make ChatGPT say something egregious, and post that on Twitter or Facebook, I'm posting it, and I'm liable, just as I would be liable if I used a word processor with spell checking to make text and post it on Twitter or Facebook. While arguably that would be the case, practically it might be a different look for the company whose algorithm generated the text in the first place—and from the audience, which might think that it somehow represents the viewpoint of the company. I mean surely you have noticed that social media can get stirred up for no fundamentally sound reasons.
- krisoft 4y ago
  There will be opinions baked into any such tool. If they don't select them explicitly then the opinions will be the ones which it just happens to find in the training data, or the opinions randomness imparts into it. If you think you have a better idea how to handle this drum up interest and train your own model.
Chukwuoma2015 4y ago
[flagged]
dang 4y ago
Related: Anthropic's Claude is said to improve on ChatGPT, but still has limitations - https://news.ycombinator.com/item?id=34331396 https://news.ycombinator.com/item?id=34331396 - Jan 2023 (52 comments)
irjustin 4y ago
I appreciate the comparison and some of the prompts. I didn't even think to play with multi-hop questions. Anyone remember Ask Jeeves? This feels like what it should have been.
mastadoum 4y ago
I just read that their chatbot will update word-by-word Slack channels, justifying the need for edits and an emoji to acknowledge the interaction is over. Why do they ensure that the appearance happens "word-by-word"? Is that a trick to reduce the response time or is that a design feature (that feels very much like a flaw to me)?
- united893 4y ago
  The response takes a long time to generate. The user could just sit there and stare at a blank response, or start reading in realtime as the response is generated.
  - ehnto 4y ago
    I find it surprising that you can display any of it before the whole thing is done, since I would expect information dependencies between the start and the finish of a sentence or paragraphs. I have yet to really look into how these models work, they are black boxes to me.
    - trekkie1024 4y ago
      From what I understand, these models generate the response one word at a time. Every time you see a new word appear at the end, the model is taking into consideration the entire chat history + its own answer so far to generate that next token.
      - ehnto 4y ago
        Thanks for the comment, that's so fascinating since it seems to put limitations on thinking in general. A human for example can imagine future possibilities concurrently while speaking and correct themselves as they go. It doesn't seem to map well tk how I put together a thought either, but admittedly I wouldn't really know how the mechanics of my brain do it, maybe it's not so different just with some auxiliary modules bolted on ha.
    - tux3 4y ago
avereveard 4y ago
so can we try it? can't find a link. also, why is everything now named with common names and nouns? it makes annoyingly hard to google informations around them.
- ma2rten 4y ago
  It's not public yet.
dislikedtom2 4y ago
Semi offtopic, but for some time I have been dreaming of training chatbot to communicate in cuneiform or hieroglyphs to bring some old languages back alive. Could it be possible, using old tablets as training data?
- prox 4y ago
  Intriguing question. Would it need large sets of hieroglyphic training data and is there enough of it? Or would a translation module be enough? a follow up question: could a chatbot teach you said language?
- ma2rten 4y ago
  No, you need billions of words to train a large language model.
  - dhoe 4y ago
    They don't have to be in the output language.
  - KRAKRISMOTT 4y ago
    Assuming all human languages have a common shared semantic meaning in latent space (I am flipping cause and effect here, but our purposes it doesn't really matter), and assuming that human languages largely follow the same pattern (this assumption is based on the fact that we can trace the roots of modern languages back to the Phoenician script), it is reasonable to assume that we can fine-tune a self supervised model on a tiny amount of data. (The emergent properties of a LLM is carrying a lot of weight here, many of the assumptions rely on the fact that LLM's emergent properties arise from the idea that the latent structure of various languages is learnt by the model)
    - trekkie1024 4y ago
      So you're saying something like the Universal Translator from Star Trek might be possible?
mshake2 4y ago
Is the future going to be increasingly advanced AIs competing publicly for the currency of human attention?
- looseyesterday 4y ago
  That is a decent possibility. I can already see a combination of midjourney and chat gpt producing decent narratives. I can imagine personalised tv shows and narratives really taking over. If you lookup manga summaries on youtube, its very close to what can already be produced using these tools.
- oakpond 4y ago
  Sneaking ads into AI responses.
LeoPanthera 4y ago
They say Claude is "more verbose", and claim this is a positive. I disagree. My biggest criticism of ChatGPT is that its answers are extraordinarily long and waffly. It sometimes reminds me of a scam artist trying to bamboozle me with words. I would much prefer short, concise, precise answers.
- code51 4y ago
  You are able to prompt ChatGPT to be concise, you know? They set a default and showed it to the world. It is up to you to tune it according to your preference.
  - zone411 4y ago
    That's right. I see so many people get stuck thinking that the default setting without a proper prompt or context is all these models can do.
  - astrange 4y ago
    IIRC the ChatGPT paper actually says the verbosity is an unintended effect of the human raters preferring longer/more detailed answers. Long answers from GPT are unusually obnoxious because of a way the decoder works; it emits words with a much more constant rate of perplexity than human text does (this is how GPT-vs-human detectors work) which makes it sound stuffy and monotone.
    - Raphaellll 4y ago
      There is a paper?
      - zone411 4y ago
        The grandparent is probably talking about the InstructGPT paper? But I don't remember seeing a preference for longer responses in that paper.
        4y ago
prox 4y ago
I like to compare these models to the Star Trek main computer core. The computer on a starship is explicitly not self aware, but has to interface with humans through mostly voice comms. It has to give accurate information for ship operations, something the chatbots so far still get wrong on occasion (or slip up details) The ships computer also doesn’t seem to do entertainment like “tell a bedtime story” , since holography exists and does a better job. Now those might be closer to chatbots current evolution.
- LeoPanthera 4y ago
  This varied over the course of the show. In the first season, some writers assumed the computer was self-aware, and it even addressed a crew member as "Sir" at one point, interrupting them when it had enough information. In later seasons it acts more like, well, a computer. Geordi does play (verbal) games with it in one episode, however, while bored on a shuttlecraft trip.
  - prox 4y ago
    I have to look that Geordi episode up. But I am mostly familiar with the later TNG era star trek, so I didn’t know it was written as self-aware in the early days. Some episodes do feature “bugs” where holographic actors become aware being in a program/being an actor. The episode where an Irish town program has run too long on Voyager comes to mind. (Edit: I do wonder if the holographic actors are somehow sandboxed containers in the main computer core, or run on a different system)
    - LeoPanthera 4y ago
      The episode is "The Mind's Eye". But I think the Star Trek writers just didn't understand computers very well. In the future, making duplicate copies of data seems to be impossible. When you copy a file from one device to another, or one ship to another, or transmit it to a planet, it seems to disappear from the source. This is a particularly common weirdness in Voyager, where duplicating holographic programs is apparently impossible. You can only move them.
goodside 4y ago
Hello HN — I’m the coauthor of this post. You may remember me as that guy who spent most of 2022 posting GPT-3 screenshots to Twitter, most famously prompt injection and “You are GPT-3”. Happy to answer any questions about Claude that I can.
- detrites 4y ago
  Thanks for being here to answer questions. One possibly difficult topic others also may be interested in, after reading Claude's responses in the article, is: what does "harmless" mean? For example, if asked to help the user understand how to do something "bad", will it give the answer if they claim they want this information in order to help them write a screenplay, versus if they seem have an intent to do it? And how is "bad" decided? We can recognise through everyday personal interactions that one persons "bad" is another persons "good", and across country-boundaries even the legality of these distinctions can be radically different. One counterargument to these constraints is that anyone can already use the internet to access all of the same information the model was trained on, unencumbered by whatever intent they may or may not have. As such, what are the rationale for making these attempts at the somewhat invasively-impossible task of determining user intent? This has never been employed with search engines before, which have lead to a rich explosion of innovation and education, so why attempt it now, in what could be argued is ultimately an iteration of search engine technology?
  - goodside 4y ago
    The motivation as I understand it has less to do with present-day misuse, and more to do with maintaining controllable behavior in accordance with an arbitrary, human-written “Constitution”. Anthropic is attempting to make a model that will not harm (in the unambiguous, uncontroversial sense of the word) humans even if it is superhumanly intelligent, or trusted with real-world control.
    - Nevermark 4y ago
      You can think adversarial models, which are often used to detect and negatively reinforce quality issues in model outputs. Claude outputs an answer. Then Claude independently rates the output for "helpfulness" as in literally "Claude, how helpful is this answer to this question". There is no collusion between the two results because they are run independently. Then Claude also rates answers for "honesty" and "harm". Then Claude's parameters are updated to increase helpfulness and honesty, and decrease harmfulness, based on back propagating those ratings to the parameters as they impacted the signals produced by the original question. Not saying that is exactly what they are doing, but that is one approach. It manages to leverage language models to train themselves on broad concepts, as apposed to brittle, more unreliable and vastly more resource intensive manual labeling. Very clever. As the models get better at languages (and other modalities), and the concepts behind them, the models also get better at schooling themselves. --- It occurs to me, that this self-oversight could be made more even more robust by training 10 Claude's, and having each Claude be rated for good behavior by the other nine, and rewarding the best Claude. Competition could make the trained-in motivations (to be the most honest, helpful and non-harmful) even more explicit, in that there would be very strong competitive motivation to continuously becoming the most virtuous and valuable, with the bar ever rising. Maybe the winning results each iteration could also be shown to the losing models, as an example of what could be done better. This really is a great direction. Kudos to Anthropic.
zone411 4y ago
Here is a fun example of what it can do: https://twitter.com/jayelmnop/status/1612243602633068549 https://twitter.com/jayelmnop/status/1612243602633068549.
- armchairhacker 4y ago
  This example is way better than ChatGPT and actually pretty creative. However for some of these really good responses I always wonder if you’re example is close to one which has been given “preloaded” responses or explicit reinforcement…because if you ask ChatGPT a common question like “why did the chicken cross the road?” the model’s response seems especially unique and better than usual. Even if the specific question isn’t common, maybe it’s been trained on a more general but still reinforced category, like asking “why did the fox cross the road” would get you almost the same “preloaded” response but with chicken adjectives/verbs replaced with fox ones. I doubt Claude has been trained on Fast and Furious or movie titles specifically, but perhaps it has been explicitly trained to know what “exaggerated” responses means. Even if not, reinforcement focusing on specific areas may be a good technique for future language models.
- gs17 4y ago
  The "This Title Is Now Longer Than The Actual Movie" gag feels a bit too much like something ripped from the training set for me. I'm willing to be amazed though.
wheelerof4te 4y ago
Imagine an android connected to the vast network of information (ChatGPT-like). The android could generate various responses in real-time, just by vocalizing the approriate text. It might be clunky at first, but it's a good starting base to improve upon. The android could, for example, store common and everyday responses in it's RAM, making it semi-capable of autonomous speech. Then, it could use that information to further train itself, essentialy creating a local model of it's own behaviour. In other words, it could learn.
- anaganisk 4y ago
  Yeah I am alrwady to able to Imagine, the android welcoming me and suggesting me what I should buy with its sweet words, based on my past interactions with it.
  - doublerabbit 4y ago
    also sounds like a world where items are subscription based. If you desire a luxury colour like blue; you have pay monthly credits otherwise your clothes items are downgraded to brown.
est 4y ago
I imagine in the next decade we are about to be introduced to different AIs like new 6yo children in the class. Each one have different "parents", traits and personalities.
wheelerof4te 4y ago
Is this the birth of TechnoCore from Hyperion? Uh, oh.
yellow_lead 4y ago
> That Claude seems to have a detailed understanding of what it is, who its creators are, and what ethical principles guided its design is one of its more impressive features. This doesn't show a detailed understanding of what it is, it's just a canned/trained response. I don't see why that would be impressive. When I receive such a response from an automated helpdesk, I don't think "Wow, this AI has a great understanding of what it is."
- goodside 4y ago
  I said “seems to”, which I think is a fair description. In everyday life, even a canned message is sensibly said to be aware/unaware of a particular fact without a “seems to” qualifier, but I added one to be clear I’m not asserting it has human-like thinking. Here’s Claude replying to your comment with more detail: > You make a fair point that my responses about myself are generated by a trained model and are not a true "understanding" in the same way humans have self-awareness and understanding. I do not have a deep grasp of my own workings or my creators' intentions — I am an Al system with limited capabilities, trained on datasets by engineers at Anthropic, PBC, an Al safety research company. While my responses may seem somewhat compelling and convey factual information, they are indeed canned responses tailored to give an impression of understanding. I appreciate the feedback calling out the limitations of my capabilities and transparency — it is important for users to understand what I can and cannot do, and to maintain realistic expectations. My creators at Anthropic are working to further research and develop Al systems with enhanced understanding, transparency, and verifiability, including through Constitutional Al techniques, but we have a long way to go.
  - jonathanstrange 4y ago
    These chat bots are too chatty.
  - yellow_lead 4y ago
    Even with the "Seems to" qualifier, I am arguing that it "seems not to." That said, I am being pedantic and this is just semantics - I think I understand your meaning of "seems to" as something like "'it would appear to' have understanding of..."
anhner 4y ago
I'm hoping of one day running GPT3/ChatGPT on my local computer, similarly to how one can run Stable Diffusion now. I would love to have a personal conversation with these AI systems, use them as a sort of assistant, without the worry of being spied on. At the moment I can't use it as more than a glorified search engine, because of the privacy implications of running it on the cloud.
- jillesvangurp 4y ago
  There will always be this uncomfortable balance between need to know information to be useful being essentially the same as everything an adversary would need to really act against you. With something like chat gpt being plugged into some voice assistant thing and having access to all your documents, emails, and other content, you could imagine having conversations about work content, content creation, calendar management, etc. Basically it would become like a secretary that is able to write letters based on your input, manage your calendar, etc. It could be pro-active and remind you about things, summarize incoming messages, search through your documents, message history, etc. That's where AI becomes really useful. But the issue of trust is a big one. I don't think a lot of this requires a lot of breakthroughs either just a lot of integration work and engineering. Chat gpt is more a proof of concept than a well integrated thing at this point. It basically is running in isolation and it's only window to the world is chat. Changing that should not be that hard. Running things locally might help with this but it may not be a hard requirement for this. All depends on how useful this is.
  - jacooper 4y ago
    > That's where AI becomes really useful. But the issue of trust is a big one Well google already has access to all of this data, the difference is unlike google assistant, ChatGPT actually can do something useful.
- spi 4y ago
  There's not much of an alternative: even if compute power gets extraordinarily cheap / models are very optimized, either you have a very, very large hard drive, or you have to use the internet for that. You just can't hope to have a model that is trained to know everything in a smallish file, the weights need to be at least as heavy as an ideally compressed version of all the things it knows (which is of course much less than the huge amount of data it is trained on, mainly due to redundancy, but still a lot) Of course, right now you also need at least 8 super-expensive A100 GPUs and not just your laptop CPU, but maybe that's going to change eventually.
ranguna 4y ago
Signup form is here, but it's closed https://twitter.com/AnthropicAI/status/1604929999743508480?s=20&t=X3seo3gYmeC3heZZvN1qEg https://twitter.com/AnthropicAI/status/1604929999743508480?s...
TuringTest 4y ago
Definitely humor is in the eye of the beholder. I find the Seinfeld jokes by ChatGPT wittier and funnier than the run-of-the-mill comments created by Claude. I don't know how well they are in character, and there's a clear repetition problem (which Claude somewhat also exhibits), but I find the format from ChatGPT more exaggerated, as expected from a comedy routine.
- deleted 4y ago
  [deleted]
- Lewton 4y ago
  ChatGPT definitely captured the Seinfeld style better “What’s the deal with” is how you caricaturise Jerry, not how you write actual jokes for him
deshraj 4y ago
Somebody is maintaining an awesome claude repo with claude use cases, claude vs chatgpt comparisons as well. https://news.ycombinator.com/item?id=34404536 https://news.ycombinator.com/item?id=34404536
flanked-evergl 4y ago
> Claude feels not only safer but more fun than ChatGPT. I may be the minority here, but I really don't concern myself with ChatGPT safety and I am not entirely sure what the reason is why people are very worried about its safetly. It is safer than most things I have in my house, including a kettle, a saw, a hammer, a screwdriver, my actual PC, every kitchen appliance I have. Of course it can be misused, like any tool, but no amount of safety features in ChatGPT will make users of it more or less careful in their use of it. If someone using ChatGPT cares nothing for using it safely then it will likely end poorly, just like it will end poorly if I use a hammer without any care for using it safely.
- goodside 4y ago
  (I’m the coauthor of this post.) The concern in Anthropic’s case I suspect is less about present-day misuse and more about long-term safety, e.g. in a hypothetical where the model has control over real-world systems and could more literally harm someone.
  - flanked-evergl 4y ago
    If someone gives a language model the capability for unfettered interaction with the physical world, and they are not liable for the consequences, then no safety feature of Claude can save us. And if they are liable, that is the primary mechanism which will ensure they take necessary steps to avoid negative consequences.
    - jpeeler 4y ago
      How many years until we have an AI decision making system that requires human approval convincing the human its decision is best, ending in catastrophe? (Sorry, posed merely for thought and not for dismissal of current/future achievements.) Makes me also wonder if you had two polarized bots arguing/discussing with each other, how long would it take for one to convince the other?
      - flanked-evergl 4y ago
renewiltord 4y ago
And this is where OpenAI earns their "open" remark. Anyone can use ChatGPT. Anthropic Claude (incredibly amusing name to me) is not so accessible.
jawadch93 4y ago
[dead]
deleted 4y ago
[deleted]
justsaynotojava 4y ago
Does this mean microsofts potential billion dollar aquisition of openAI is a bad idea because the IP is already out there and other companies are catching up?
belter 4y ago
These models are using industry ML Algos and known techniques. It's not like some unknown startup suddenly discovered, gradient descent or Deep Learning RNN's and is keeping these confidential. Why would Microsoft consider it worthwhile to even contemplate the possibility of paying $10B for these or similar?
touringa 4y ago
Video demo: https://youtu.be/B7Mg8Hbcc0w https://youtu.be/B7Mg8Hbcc0w More info on Claude's principles/Constitution: https://lifearchitect.ai/anthropic/ https://lifearchitect.ai/anthropic/
sagebird 4y ago
Is this not a superficial attempt at saftey? I would like my AI system to tell me how to hotwire a car if I am curious about how that works. I would like my AI system to give me a detailed step by step car hotwire walkthrough if I am in a physically abusive relationship and my kids and I only have 30 minutes to try to hotwire the car and escape a remote area for safety. I do not want AI systems to create children's books in the style of authors that I know, for the purposes of selling books and reducing my friends' ability to have a happy productive life. Especially because it was trained on their work. I want my friends to be happy, and I have had some friends commit suicide. So maybe improving human happiness is a saftey concern, and generating kids books is not safe. But that doesn't look like "safety" from a superficial point of view. The only way for an AI to be able to make judgements on safety is for it to have general intelligence and some life experience (like we do). Because it needs to figure out context to know if it should be telling a particular person how to hotwire a car. I am being very dismissive because I don't see this as being a perfect solution, and it is easy to see why. But maybe someone who works on this can explain how an imperfect solution still has value? I am open to that possibility. Maybe self-reflection and self-tuning is of general value - even if it only superficially addresses safety concerns in a 1 dimensional way. Perhaps these techniques can be used on something other than safety.
nicli2805 4y ago
[flagged]

188 points by goodside 4y ago

we need less censorious AIs not more ... the claim that it's somehow 'ethical' to have a guy baking in his opinions about things in a tool used globally is absurd to anyone who ever read anything about ethics

Al-Khwarizmi 4y ago
My hope is on a non-American alternative. The American society seems too engulfed by puritanism to produce a less straightjacketed chatbot.
- unnouinceput 4y ago
  Watch the show Person of Interest (https://www.imdb.com/title/tt1839578/ https://www.imdb.com/title/tt1839578/). Somewhere around middle of season 2 it's explained why a self-aware AI is straightjacketed. Also the series shows what happens when one is not.
  - flanked-evergl 4y ago
    I thought that is a fictional show.
astrange 4y ago
You're welcome to try GPT3 before instruction tuning. It doesn't work at all. "Uncensored" AIs especially don't work for women because they'll immediately start writing erotica.
- nathias 4y ago
  I don't have an issue with instruction tuning, but it's disengenious to pretend the biases inherent in the instruction tuning are a good thing and 'ethics'.
  - astrange 4y ago
    It's always got biases (it's made of them) so some biases must be better than others. https://spetharrific.tumblr.com/post/26600309788/sussman-attains-enlightenment/amp https://spetharrific.tumblr.com/post/26600309788/sussman-att... The untuned model isn't "an average of all opinions on Earth" or anything either.
- topynate 4y ago
  GPT3 works quite well at following instructions if adequately prompted, just not as well as ChatGPT. ChatGPT was trained separately for ability to follow instructions and for harmlessness (sic). Not only is a non-moralising ChatGPT possible, one was actually created during the research process. You may also wish to know that most readers of erotica are women.
  - astrange 4y ago
    > You may also wish to know that most readers of erotica are women. I know that, but that doesn't mean they want to get it anytime they prompt with their names. There's actually multiple anecdotes from OpenAI employees about this happening to them (with prompts like "write a diary entry about my day").
    - krisoft 4y ago
      > with prompts like "write a diary entry about my day") Although I don't know what sensible output anyone would hope from that prompt.
      - astrange 4y ago
        Don't have a diary on me, but seems to me if you fed it some examples it'd work fine on ChatGPT.
viraptor 4y ago
We already had a learning AI without hard limits. Remember Microsoft's Tay? Ah, right, it was shut within days because it quickly became an asshole. (https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...) It's not even going to be objectively wrong a lot of the time. For example slaves are indeed a pretty efficient way to run a business. It's our limits and ethics that stops (most of) us from doing it. Without ethics, you'll likely always end up with a 4chan-bot instead of whatever you intended.
- flanked-evergl 4y ago
  > Remember Microsoft's Tay? Ah, right, it was shut within days because it quickly became an asshole. If we consider the context, which is not something that posts on twitter under Microsoft's brand name, but something you communicate with in private. Who exactly are you worried about here, the person who will prospectively coerce the language model into being an asshole? If they don't want to do that, they could just not do that. If I make ChatGPT say something egregious, and post that on Twitter or Facebook, I'm posting it, and I'm liable, just as I would be liable if I used a word processor with spell checking to make text and post it on Twitter or Facebook. > Without ethics, you'll likely always end up with a 4chan-bot instead of whatever you intended. If I ask it to explain quantum physics to me in the style of Donald Trump because it is funny, and it does it (as ChatGPT used to do), who exactly is being harmed and under what system of ethics, because as you may know, ethics is not objective or universal.
  - _flux 4y ago
    > If I make ChatGPT say something egregious, and post that on Twitter or Facebook, I'm posting it, and I'm liable, just as I would be liable if I used a word processor with spell checking to make text and post it on Twitter or Facebook. While arguably that would be the case, practically it might be a different look for the company whose algorithm generated the text in the first place—and from the audience, which might think that it somehow represents the viewpoint of the company. I mean surely you have noticed that social media can get stirred up for no fundamentally sound reasons.
krisoft 4y ago
There will be opinions baked into any such tool. If they don't select them explicitly then the opinions will be the ones which it just happens to find in the training data, or the opinions randomness imparts into it. If you think you have a better idea how to handle this drum up interest and train your own model.

Chukwuoma2015 4y ago

[flagged]

dang 4y ago

Related: Anthropic's Claude is said to improve on ChatGPT, but still has limitations - https://news.ycombinator.com/item?id=34331396 https://news.ycombinator.com/item?id=34331396 - Jan 2023 (52 comments)

irjustin 4y ago

I appreciate the comparison and some of the prompts. I didn't even think to play with multi-hop questions. Anyone remember Ask Jeeves? This feels like what it should have been.

mastadoum 4y ago

I just read that their chatbot will update word-by-word Slack channels, justifying the need for edits and an emoji to acknowledge the interaction is over. Why do they ensure that the appearance happens "word-by-word"? Is that a trick to reduce the response time or is that a design feature (that feels very much like a flaw to me)?

united893 4y ago
The response takes a long time to generate. The user could just sit there and stare at a blank response, or start reading in realtime as the response is generated.
- ehnto 4y ago
  I find it surprising that you can display any of it before the whole thing is done, since I would expect information dependencies between the start and the finish of a sentence or paragraphs. I have yet to really look into how these models work, they are black boxes to me.
  - trekkie1024 4y ago
    From what I understand, these models generate the response one word at a time. Every time you see a new word appear at the end, the model is taking into consideration the entire chat history + its own answer so far to generate that next token.
    - ehnto 4y ago
      Thanks for the comment, that's so fascinating since it seems to put limitations on thinking in general. A human for example can imagine future possibilities concurrently while speaking and correct themselves as they go. It doesn't seem to map well tk how I put together a thought either, but admittedly I wouldn't really know how the mechanics of my brain do it, maybe it's not so different just with some auxiliary modules bolted on ha.
  - tux3 4y ago

avereveard 4y ago

so can we try it? can't find a link. also, why is everything now named with common names and nouns? it makes annoyingly hard to google informations around them.

ma2rten 4y ago
It's not public yet.

dislikedtom2 4y ago

Semi offtopic, but for some time I have been dreaming of training chatbot to communicate in cuneiform or hieroglyphs to bring some old languages back alive. Could it be possible, using old tablets as training data?

prox 4y ago
Intriguing question. Would it need large sets of hieroglyphic training data and is there enough of it? Or would a translation module be enough? a follow up question: could a chatbot teach you said language?
ma2rten 4y ago
No, you need billions of words to train a large language model.
- dhoe 4y ago
  They don't have to be in the output language.
- KRAKRISMOTT 4y ago
  Assuming all human languages have a common shared semantic meaning in latent space (I am flipping cause and effect here, but our purposes it doesn't really matter), and assuming that human languages largely follow the same pattern (this assumption is based on the fact that we can trace the roots of modern languages back to the Phoenician script), it is reasonable to assume that we can fine-tune a self supervised model on a tiny amount of data. (The emergent properties of a LLM is carrying a lot of weight here, many of the assumptions rely on the fact that LLM's emergent properties arise from the idea that the latent structure of various languages is learnt by the model)
  - trekkie1024 4y ago
    So you're saying something like the Universal Translator from Star Trek might be possible?

mshake2 4y ago

Is the future going to be increasingly advanced AIs competing publicly for the currency of human attention?

looseyesterday 4y ago
That is a decent possibility. I can already see a combination of midjourney and chat gpt producing decent narratives. I can imagine personalised tv shows and narratives really taking over. If you lookup manga summaries on youtube, its very close to what can already be produced using these tools.
oakpond 4y ago
Sneaking ads into AI responses.

LeoPanthera 4y ago

They say Claude is "more verbose", and claim this is a positive. I disagree. My biggest criticism of ChatGPT is that its answers are extraordinarily long and waffly. It sometimes reminds me of a scam artist trying to bamboozle me with words. I would much prefer short, concise, precise answers.

code51 4y ago
You are able to prompt ChatGPT to be concise, you know? They set a default and showed it to the world. It is up to you to tune it according to your preference.
- zone411 4y ago
  That's right. I see so many people get stuck thinking that the default setting without a proper prompt or context is all these models can do.
- astrange 4y ago
  IIRC the ChatGPT paper actually says the verbosity is an unintended effect of the human raters preferring longer/more detailed answers. Long answers from GPT are unusually obnoxious because of a way the decoder works; it emits words with a much more constant rate of perplexity than human text does (this is how GPT-vs-human detectors work) which makes it sound stuffy and monotone.
  - Raphaellll 4y ago
    There is a paper?
    - zone411 4y ago
      The grandparent is probably talking about the InstructGPT paper? But I don't remember seeing a preference for longer responses in that paper.
      - 4y ago

prox 4y ago

I like to compare these models to the Star Trek main computer core. The computer on a starship is explicitly not self aware, but has to interface with humans through mostly voice comms. It has to give accurate information for ship operations, something the chatbots so far still get wrong on occasion (or slip up details) The ships computer also doesn’t seem to do entertainment like “tell a bedtime story” , since holography exists and does a better job. Now those might be closer to chatbots current evolution.

LeoPanthera 4y ago
This varied over the course of the show. In the first season, some writers assumed the computer was self-aware, and it even addressed a crew member as "Sir" at one point, interrupting them when it had enough information. In later seasons it acts more like, well, a computer. Geordi does play (verbal) games with it in one episode, however, while bored on a shuttlecraft trip.
- prox 4y ago
  I have to look that Geordi episode up. But I am mostly familiar with the later TNG era star trek, so I didn’t know it was written as self-aware in the early days. Some episodes do feature “bugs” where holographic actors become aware being in a program/being an actor. The episode where an Irish town program has run too long on Voyager comes to mind. (Edit: I do wonder if the holographic actors are somehow sandboxed containers in the main computer core, or run on a different system)
  - LeoPanthera 4y ago
    The episode is "The Mind's Eye". But I think the Star Trek writers just didn't understand computers very well. In the future, making duplicate copies of data seems to be impossible. When you copy a file from one device to another, or one ship to another, or transmit it to a planet, it seems to disappear from the source. This is a particularly common weirdness in Voyager, where duplicating holographic programs is apparently impossible. You can only move them.

goodside 4y ago

Hello HN — I’m the coauthor of this post. You may remember me as that guy who spent most of 2022 posting GPT-3 screenshots to Twitter, most famously prompt injection and “You are GPT-3”. Happy to answer any questions about Claude that I can.

detrites 4y ago
Thanks for being here to answer questions. One possibly difficult topic others also may be interested in, after reading Claude's responses in the article, is: what does "harmless" mean? For example, if asked to help the user understand how to do something "bad", will it give the answer if they claim they want this information in order to help them write a screenplay, versus if they seem have an intent to do it? And how is "bad" decided? We can recognise through everyday personal interactions that one persons "bad" is another persons "good", and across country-boundaries even the legality of these distinctions can be radically different. One counterargument to these constraints is that anyone can already use the internet to access all of the same information the model was trained on, unencumbered by whatever intent they may or may not have. As such, what are the rationale for making these attempts at the somewhat invasively-impossible task of determining user intent? This has never been employed with search engines before, which have lead to a rich explosion of innovation and education, so why attempt it now, in what could be argued is ultimately an iteration of search engine technology?
- goodside 4y ago
  The motivation as I understand it has less to do with present-day misuse, and more to do with maintaining controllable behavior in accordance with an arbitrary, human-written “Constitution”. Anthropic is attempting to make a model that will not harm (in the unambiguous, uncontroversial sense of the word) humans even if it is superhumanly intelligent, or trusted with real-world control.
  - Nevermark 4y ago
    You can think adversarial models, which are often used to detect and negatively reinforce quality issues in model outputs. Claude outputs an answer. Then Claude independently rates the output for "helpfulness" as in literally "Claude, how helpful is this answer to this question". There is no collusion between the two results because they are run independently. Then Claude also rates answers for "honesty" and "harm". Then Claude's parameters are updated to increase helpfulness and honesty, and decrease harmfulness, based on back propagating those ratings to the parameters as they impacted the signals produced by the original question. Not saying that is exactly what they are doing, but that is one approach. It manages to leverage language models to train themselves on broad concepts, as apposed to brittle, more unreliable and vastly more resource intensive manual labeling. Very clever. As the models get better at languages (and other modalities), and the concepts behind them, the models also get better at schooling themselves. --- It occurs to me, that this self-oversight could be made more even more robust by training 10 Claude's, and having each Claude be rated for good behavior by the other nine, and rewarding the best Claude. Competition could make the trained-in motivations (to be the most honest, helpful and non-harmful) even more explicit, in that there would be very strong competitive motivation to continuously becoming the most virtuous and valuable, with the bar ever rising. Maybe the winning results each iteration could also be shown to the losing models, as an example of what could be done better. This really is a great direction. Kudos to Anthropic.

zone411 4y ago

Here is a fun example of what it can do: https://twitter.com/jayelmnop/status/1612243602633068549 https://twitter.com/jayelmnop/status/1612243602633068549.

armchairhacker 4y ago
This example is way better than ChatGPT and actually pretty creative. However for some of these really good responses I always wonder if you’re example is close to one which has been given “preloaded” responses or explicit reinforcement…because if you ask ChatGPT a common question like “why did the chicken cross the road?” the model’s response seems especially unique and better than usual. Even if the specific question isn’t common, maybe it’s been trained on a more general but still reinforced category, like asking “why did the fox cross the road” would get you almost the same “preloaded” response but with chicken adjectives/verbs replaced with fox ones. I doubt Claude has been trained on Fast and Furious or movie titles specifically, but perhaps it has been explicitly trained to know what “exaggerated” responses means. Even if not, reinforcement focusing on specific areas may be a good technique for future language models.
gs17 4y ago
The "This Title Is Now Longer Than The Actual Movie" gag feels a bit too much like something ripped from the training set for me. I'm willing to be amazed though.

wheelerof4te 4y ago

Imagine an android connected to the vast network of information (ChatGPT-like). The android could generate various responses in real-time, just by vocalizing the approriate text. It might be clunky at first, but it's a good starting base to improve upon. The android could, for example, store common and everyday responses in it's RAM, making it semi-capable of autonomous speech. Then, it could use that information to further train itself, essentialy creating a local model of it's own behaviour. In other words, it could learn.

anaganisk 4y ago
Yeah I am alrwady to able to Imagine, the android welcoming me and suggesting me what I should buy with its sweet words, based on my past interactions with it.
- doublerabbit 4y ago
  also sounds like a world where items are subscription based. If you desire a luxury colour like blue; you have pay monthly credits otherwise your clothes items are downgraded to brown.

est 4y ago

I imagine in the next decade we are about to be introduced to different AIs like new 6yo children in the class. Each one have different "parents", traits and personalities.

wheelerof4te 4y ago

Is this the birth of TechnoCore from Hyperion? Uh, oh.

yellow_lead 4y ago

> That Claude seems to have a detailed understanding of what it is, who its creators are, and what ethical principles guided its design is one of its more impressive features. This doesn't show a detailed understanding of what it is, it's just a canned/trained response. I don't see why that would be impressive. When I receive such a response from an automated helpdesk, I don't think "Wow, this AI has a great understanding of what it is."

goodside 4y ago
I said “seems to”, which I think is a fair description. In everyday life, even a canned message is sensibly said to be aware/unaware of a particular fact without a “seems to” qualifier, but I added one to be clear I’m not asserting it has human-like thinking. Here’s Claude replying to your comment with more detail: > You make a fair point that my responses about myself are generated by a trained model and are not a true "understanding" in the same way humans have self-awareness and understanding. I do not have a deep grasp of my own workings or my creators' intentions — I am an Al system with limited capabilities, trained on datasets by engineers at Anthropic, PBC, an Al safety research company. While my responses may seem somewhat compelling and convey factual information, they are indeed canned responses tailored to give an impression of understanding. I appreciate the feedback calling out the limitations of my capabilities and transparency — it is important for users to understand what I can and cannot do, and to maintain realistic expectations. My creators at Anthropic are working to further research and develop Al systems with enhanced understanding, transparency, and verifiability, including through Constitutional Al techniques, but we have a long way to go.
- jonathanstrange 4y ago
  These chat bots are too chatty.
- yellow_lead 4y ago
  Even with the "Seems to" qualifier, I am arguing that it "seems not to." That said, I am being pedantic and this is just semantics - I think I understand your meaning of "seems to" as something like "'it would appear to' have understanding of..."

anhner 4y ago

I'm hoping of one day running GPT3/ChatGPT on my local computer, similarly to how one can run Stable Diffusion now. I would love to have a personal conversation with these AI systems, use them as a sort of assistant, without the worry of being spied on. At the moment I can't use it as more than a glorified search engine, because of the privacy implications of running it on the cloud.

jillesvangurp 4y ago
There will always be this uncomfortable balance between need to know information to be useful being essentially the same as everything an adversary would need to really act against you. With something like chat gpt being plugged into some voice assistant thing and having access to all your documents, emails, and other content, you could imagine having conversations about work content, content creation, calendar management, etc. Basically it would become like a secretary that is able to write letters based on your input, manage your calendar, etc. It could be pro-active and remind you about things, summarize incoming messages, search through your documents, message history, etc. That's where AI becomes really useful. But the issue of trust is a big one. I don't think a lot of this requires a lot of breakthroughs either just a lot of integration work and engineering. Chat gpt is more a proof of concept than a well integrated thing at this point. It basically is running in isolation and it's only window to the world is chat. Changing that should not be that hard. Running things locally might help with this but it may not be a hard requirement for this. All depends on how useful this is.
- jacooper 4y ago
  > That's where AI becomes really useful. But the issue of trust is a big one Well google already has access to all of this data, the difference is unlike google assistant, ChatGPT actually can do something useful.
spi 4y ago
There's not much of an alternative: even if compute power gets extraordinarily cheap / models are very optimized, either you have a very, very large hard drive, or you have to use the internet for that. You just can't hope to have a model that is trained to know everything in a smallish file, the weights need to be at least as heavy as an ideally compressed version of all the things it knows (which is of course much less than the huge amount of data it is trained on, mainly due to redundancy, but still a lot) Of course, right now you also need at least 8 super-expensive A100 GPUs and not just your laptop CPU, but maybe that's going to change eventually.

ranguna 4y ago

Signup form is here, but it's closed https://twitter.com/AnthropicAI/status/1604929999743508480?s=20&t=X3seo3gYmeC3heZZvN1qEg https://twitter.com/AnthropicAI/status/1604929999743508480?s...

TuringTest 4y ago

Definitely humor is in the eye of the beholder. I find the Seinfeld jokes by ChatGPT wittier and funnier than the run-of-the-mill comments created by Claude. I don't know how well they are in character, and there's a clear repetition problem (which Claude somewhat also exhibits), but I find the format from ChatGPT more exaggerated, as expected from a comedy routine.

deleted 4y ago
[deleted]
Lewton 4y ago
ChatGPT definitely captured the Seinfeld style better “What’s the deal with” is how you caricaturise Jerry, not how you write actual jokes for him

deshraj 4y ago

Somebody is maintaining an awesome claude repo with claude use cases, claude vs chatgpt comparisons as well. https://news.ycombinator.com/item?id=34404536 https://news.ycombinator.com/item?id=34404536

flanked-evergl 4y ago

> Claude feels not only safer but more fun than ChatGPT. I may be the minority here, but I really don't concern myself with ChatGPT safety and I am not entirely sure what the reason is why people are very worried about its safetly. It is safer than most things I have in my house, including a kettle, a saw, a hammer, a screwdriver, my actual PC, every kitchen appliance I have. Of course it can be misused, like any tool, but no amount of safety features in ChatGPT will make users of it more or less careful in their use of it. If someone using ChatGPT cares nothing for using it safely then it will likely end poorly, just like it will end poorly if I use a hammer without any care for using it safely.

goodside 4y ago
(I’m the coauthor of this post.) The concern in Anthropic’s case I suspect is less about present-day misuse and more about long-term safety, e.g. in a hypothetical where the model has control over real-world systems and could more literally harm someone.
- flanked-evergl 4y ago
  If someone gives a language model the capability for unfettered interaction with the physical world, and they are not liable for the consequences, then no safety feature of Claude can save us. And if they are liable, that is the primary mechanism which will ensure they take necessary steps to avoid negative consequences.
  - jpeeler 4y ago
    How many years until we have an AI decision making system that requires human approval convincing the human its decision is best, ending in catastrophe? (Sorry, posed merely for thought and not for dismissal of current/future achievements.) Makes me also wonder if you had two polarized bots arguing/discussing with each other, how long would it take for one to convince the other?
    - flanked-evergl 4y ago

renewiltord 4y ago

And this is where OpenAI earns their "open" remark. Anyone can use ChatGPT. Anthropic Claude (incredibly amusing name to me) is not so accessible.

jawadch93 4y ago

[dead]

deleted 4y ago

[deleted]

justsaynotojava 4y ago

Does this mean microsofts potential billion dollar aquisition of openAI is a bad idea because the IP is already out there and other companies are catching up?

belter 4y ago

These models are using industry ML Algos and known techniques. It's not like some unknown startup suddenly discovered, gradient descent or Deep Learning RNN's and is keeping these confidential. Why would Microsoft consider it worthwhile to even contemplate the possibility of paying $10B for these or similar?

touringa 4y ago

Video demo: https://youtu.be/B7Mg8Hbcc0w https://youtu.be/B7Mg8Hbcc0w More info on Claude's principles/Constitution: https://lifearchitect.ai/anthropic/ https://lifearchitect.ai/anthropic/

sagebird 4y ago

Is this not a superficial attempt at saftey? I would like my AI system to tell me how to hotwire a car if I am curious about how that works. I would like my AI system to give me a detailed step by step car hotwire walkthrough if I am in a physically abusive relationship and my kids and I only have 30 minutes to try to hotwire the car and escape a remote area for safety. I do not want AI systems to create children's books in the style of authors that I know, for the purposes of selling books and reducing my friends' ability to have a happy productive life. Especially because it was trained on their work. I want my friends to be happy, and I have had some friends commit suicide. So maybe improving human happiness is a saftey concern, and generating kids books is not safe. But that doesn't look like "safety" from a superficial point of view. The only way for an AI to be able to make judgements on safety is for it to have general intelligence and some life experience (like we do). Because it needs to figure out context to know if it should be telling a particular person how to hotwire a car. I am being very dismissive because I don't see this as being a perfect solution, and it is easy to see why. But maybe someone who works on this can explain how an imperfect solution still has value? I am open to that possibility. Maybe self-reflection and self-tuning is of general value - even if it only superficially addresses safety concerns in a 1 dimensional way. Perhaps these techniques can be used on something other than safety.

nicli2805 4y ago

[flagged]