6 ms·

Claude 2 Internal API Client and CLI

84 points by explosion-s 3y ago

ronsor 3y ago
This client wouldn't exist if it were possible to actually get access to the official API.
- linsomniac 3y ago
  Have you tried getting on the waitlist? It worked for me, ISTR it took around 2 weeks.
  - ronsor 3y ago
    I'm pretty sure it's been over a month now since I submitted my application
  - williamstein 3y ago
    I submitted applications three times to their waitlist over the last several months, and I haver never heard back with any response at all. I think my use case is very reasonable (integration with https://cocalc.com https://cocalc.com, where we use ChatGPT's API heavily right now). My experience is that you fill out a web form to request access to the waitlist, and get no feedback at all ever (I just double checked my spam folders as well). Is that what normally happens for people?
    - williamstein 3y ago
      The magic of Hacker News - I just got an invite :-)
      - npsomaratna 3y ago
        If someone from Anthropic is reading this thread: do consider me too ;) I've submitted 3 applications over a period of several months so far. Complete radio silence. At this point, Anthropic completely ignoring waitlist applications (without even a confirmation) is almost a meme. By the time they open up, a lot of devs will be entrenched in OpenAI's arms.
  - PartiallyTyped 3y ago
    I have been on it for months. Perhaps it's a country problem?
  - nkko 3y ago
    I have tried a few times with private and company emails from different countries, of which is USA, with no luck.
- explosion-s 3y ago
  The API costs money though
dandiep 3y ago
Who here is using Claude? And can you comment on your experiences with it vs. GPT 3.5/4?
- BryanLegend 3y ago
  I'm pleased with it. Claude seems kinder and less patronizing than GPT. Not as good at coding yet.
- xfalcox 3y ago
  I added it as an option in Discourse, and I've been happy with it's output for summarization tasks, suggesting titles and proofreading.
- jerrygenser 3y ago
  It's good for general things but less good at coding. Can usually get the correct answer for simpler things but much less idiomatic for python than gpt4
  - ec109685 3y ago
    For javascript, it did just as well as gpt-4 for several questions and used more modern JavaScript syntax. First time I have felt something feel nearly as good, and the user interface is a bit nicer.
    - speedgoose 3y ago
      Does it uses the ECMAScript modules instead of the CommonJS modules by default?
      - ec109685 3y ago
        No, but it tends to use const more and the code looks more natural whereas gpt-4 has a single operation per line (but maybe gpt-4 easier to follow). For both you can prompt it to use module syntax. I am excited about GitHub Copilot X given you’ll get the best of ChatGPG and Copilot, making the tool much more useful.
bmitc 3y ago
I think it would be nice if companies and projects stopped using famous names to promote their projects.
- catgary 3y ago
  Yeah I rolled my eyes pretty hard when a crypto company used something like “team grothendieck”.
- refulgentis 3y ago
  Claude Shannon. It's a beautiful homage.
- capableweb 3y ago
  I wholeheartedly agree! I think Van Damme is macho enough to let it slide for this time though, at least I hope so, otherwise the project is surely in danger.
  - bmitc 3y ago
    That's not as clever as you think. :)
cubefox 3y ago
Note that Claude 2 scores 71.2% zero-shot on the python coding benchmark HumanEval, which is better than GPT-4, which scores 67.0%. Is there already real-world experience with its programming performance?
- lerchmo 3y ago
  I have found just using it in the web interface comperable to OpenAI. But the context window makes a huge difference. I can dump alot more files in ( entire schema, sample records etc)
- famouswaffles 3y ago
  GPT-4 out in the wild's (reproducible) performance appears to be much higher than 67. Testing from 3/15 (presumably on the 0314 model) seems to be at 85.36% (https://twitter.com/amanrsanger/status/1635751764577361921 https://twitter.com/amanrsanger/status/1635751764577361921). And the linked paper from my post(https://doi.org/10.48550/arXiv.2305.01210 https://doi.org/10.48550/arXiv.2305.01210) got a pass@1 of 88.4 from GPT-4 recently (May? June?).
dang 3y ago
Recent and related: Claude 2 - https://news.ycombinator.com/item?id=36680755 https://news.ycombinator.com/item?id=36680755 - July 2023 (255 comments) Model card and evaluations for Claude models [pdf] - https://news.ycombinator.com/item?id=36681982 https://news.ycombinator.com/item?id=36681982 - July 2023 (25 comments)
anotherpaulg 3y ago
I've been on the API waitlist for months. I'd like to integrate Claude in my open source AI coding assistant tool. I've had numerous users request this over the past couple of months: https://github.com/paul-gauthier/aider/issues/7 https://github.com/paul-gauthier/aider/issues/7 Feels like it would be unwise to build atop something unofficial like this?

84 points by explosion-s 3y ago

ronsor 3y ago

This client wouldn't exist if it were possible to actually get access to the official API.

linsomniac 3y ago
Have you tried getting on the waitlist? It worked for me, ISTR it took around 2 weeks.
- ronsor 3y ago
  I'm pretty sure it's been over a month now since I submitted my application
- williamstein 3y ago
  I submitted applications three times to their waitlist over the last several months, and I haver never heard back with any response at all. I think my use case is very reasonable (integration with https://cocalc.com https://cocalc.com, where we use ChatGPT's API heavily right now). My experience is that you fill out a web form to request access to the waitlist, and get no feedback at all ever (I just double checked my spam folders as well). Is that what normally happens for people?
  - williamstein 3y ago
    The magic of Hacker News - I just got an invite :-)
    - npsomaratna 3y ago
      If someone from Anthropic is reading this thread: do consider me too ;) I've submitted 3 applications over a period of several months so far. Complete radio silence. At this point, Anthropic completely ignoring waitlist applications (without even a confirmation) is almost a meme. By the time they open up, a lot of devs will be entrenched in OpenAI's arms.
- PartiallyTyped 3y ago
  I have been on it for months. Perhaps it's a country problem?
- nkko 3y ago
  I have tried a few times with private and company emails from different countries, of which is USA, with no luck.
explosion-s 3y ago
The API costs money though

dandiep 3y ago

Who here is using Claude? And can you comment on your experiences with it vs. GPT 3.5/4?

BryanLegend 3y ago
I'm pleased with it. Claude seems kinder and less patronizing than GPT. Not as good at coding yet.
xfalcox 3y ago
I added it as an option in Discourse, and I've been happy with it's output for summarization tasks, suggesting titles and proofreading.
jerrygenser 3y ago
It's good for general things but less good at coding. Can usually get the correct answer for simpler things but much less idiomatic for python than gpt4
- ec109685 3y ago
  For javascript, it did just as well as gpt-4 for several questions and used more modern JavaScript syntax. First time I have felt something feel nearly as good, and the user interface is a bit nicer.
  - speedgoose 3y ago
    Does it uses the ECMAScript modules instead of the CommonJS modules by default?
    - ec109685 3y ago
      No, but it tends to use const more and the code looks more natural whereas gpt-4 has a single operation per line (but maybe gpt-4 easier to follow). For both you can prompt it to use module syntax. I am excited about GitHub Copilot X given you’ll get the best of ChatGPG and Copilot, making the tool much more useful.

bmitc 3y ago

I think it would be nice if companies and projects stopped using famous names to promote their projects.

catgary 3y ago
Yeah I rolled my eyes pretty hard when a crypto company used something like “team grothendieck”.
refulgentis 3y ago
Claude Shannon. It's a beautiful homage.
capableweb 3y ago
I wholeheartedly agree! I think Van Damme is macho enough to let it slide for this time though, at least I hope so, otherwise the project is surely in danger.
- bmitc 3y ago
  That's not as clever as you think. :)

cubefox 3y ago

Note that Claude 2 scores 71.2% zero-shot on the python coding benchmark HumanEval, which is better than GPT-4, which scores 67.0%. Is there already real-world experience with its programming performance?

lerchmo 3y ago
I have found just using it in the web interface comperable to OpenAI. But the context window makes a huge difference. I can dump alot more files in ( entire schema, sample records etc)
famouswaffles 3y ago
GPT-4 out in the wild's (reproducible) performance appears to be much higher than 67. Testing from 3/15 (presumably on the 0314 model) seems to be at 85.36% (https://twitter.com/amanrsanger/status/1635751764577361921 https://twitter.com/amanrsanger/status/1635751764577361921). And the linked paper from my post(https://doi.org/10.48550/arXiv.2305.01210 https://doi.org/10.48550/arXiv.2305.01210) got a pass@1 of 88.4 from GPT-4 recently (May? June?).

dang 3y ago

Recent and related: Claude 2 - https://news.ycombinator.com/item?id=36680755 https://news.ycombinator.com/item?id=36680755 - July 2023 (255 comments) Model card and evaluations for Claude models [pdf] - https://news.ycombinator.com/item?id=36681982 https://news.ycombinator.com/item?id=36681982 - July 2023 (25 comments)

anotherpaulg 3y ago

I've been on the API waitlist for months. I'd like to integrate Claude in my open source AI coding assistant tool. I've had numerous users request this over the past couple of months: https://github.com/paul-gauthier/aider/issues/7 https://github.com/paul-gauthier/aider/issues/7 Feels like it would be unwise to build atop something unofficial like this?