8 ms·

ChatGPT Now Losing Users

48 points by noufalibrahim 3y ago

tmaly 3y ago
I have seen it posted before, SimilarWeb numbers are questionable. If OpenAI can publish user stats, I believe the claim as long as they are open about their methodology.
damnesian 3y ago
>Have we hit peak AI? AI is at the same point cinema was at when pioneers of the genre showed short films in "salons" designed to frighten people with simple tricks. This is hardly the peak. We're just at the beginning.
- orwin 3y ago
  AI development is different than most CS fields, as you need a solid mathematical base to build a "new discovery", and a lot of computation to use it efficiently. I'm 100% certain it's not the AI peak, but i'm also confident that we are close to the local maxima, at least for transformer architecture. Maybe the following improvement will be in chaining architecture, or a brand new technique, or maybe progress will be made in parallel fields (even smaller processors!). Charitably, i'll decide that the question meant "Have we hit peak GPT" and in this case, this is a legitimate question.
  - unshavedyak 3y ago
    I'm most interested in refined training sets, refined compression, etc. Where do you suspect the cap is there? I say that because i find GPT4 extremely useful. Even if GPT4 (in it's hypothetical un-nerfed state some would argue)* is the peak LLM i think we have a ton of room to grow even within that peak. Locally running larger context windows, multiple domain experts in parallel and hooked up to a local repository of available IO actions would give a very significant "advancement" within the same conceptual "peak LLM". Now will we be able to get there on consumer hardware? Not idea. However i don't think i need more than GPT4 to find the whole technology extremely useful. Moderately cheap to run large context windows[1] entirely locally is the "next big thing" for me. I get a fair bit of value from GPT4 and Phind. The technology has sold me already. [1]: I should amend, i'm speaking loosely about context windows. What i mean is the ability to feed it refined datasets locally to have it's "local knowledge" change, which may be less about context windows and more about refined training. I'll leave that to the experts to debate, i am assuredly nothing close.
    - orwin 3y ago
      > I say that because i find GPT4 extremely useful. Even if GPT4 (in it's hypothetical un-nerfed state some would argue)* is the peak LLM i think we have a ton of room to grow even within that peak. Yes. sorry my point wasn't well formulated (i'm not an expert either: i've trained LLMs on receipts in 2016, and quickly decided i hated that). I'll try doing it better by dividing the two idea i put in that post: - i think that GPT4 is probably close the peak generalist GPT, meaning that maybe Bard can surpass it (or a chatGPT4.5 or something) but the improvement won't be stellar. - If future improvement won't come from the transformer architecture itself (i absolutely can be wrong on that), it'll come from what we interface with it. Absolutely context windows will probably a great way to have "specialists" transformers, but alos linking it with different "kind" of AI might push it too (i've read a post from Wolfram a while ago that sold me on interlinking different AI models/languages/state machine). I'm sure specialist or even large users haves tons of ideas. I got some value from GPT3.5, enough for me to buy access to GPT4. with a caveat: i'm pretty sure that for work-related questions i lost a bit more time than i "gained" overall. because my mastery of the technologies we use and very specific needs that GPT cannot know. But in the last three weeks or so, i'm pretty sure it was overall a net benefit at work, as i know better when i should use it.
  - CuriouslyC 3y ago
    I don't think we'll need to completely replace the transformer, we just need better, more annotated data sets and a really good solution to large context windows. Data set annotation has a huge impact on model output, you can see this for yourself by comparing SDXL to Stable Diffusion 1.5, the model is larger but the real juice comes from using aesthetics scores for images during training to bias output towards things that look good - compare it to other large models like DALL-E or Imagen and you'll see parameter size is not the most important thing. With larger context windows, we can use traditional search algorithms to cherry pick an educational context for the LLM for few shot learning. If that's paired with more, better and annotated data, that will take us quite far.
- YeGoblynQueenne 3y ago
  We're not at the beginning. AI research started in 1956 with the Dartmouth workshop [1]. We've already had several cycles of boom and bust and at least two AI winters already [2]. Btw, there was a sci-fi novel where the plot took place on a planet where winter came in periods too long for the natives of the planet to remember (they had short lifespans, or they had limited memories, I er, why, I can't remember!). I don't remember the title of the novel or the author and I've been trying to find it. I thought it was something by Ursula K. Le Guin, but it doesn't sound like it. And no, it's not Game of Thrones. I think GoT took the idea of long-coming winters from the earlier novel, or it's just a coincidence (or cryptomnesia). Anyway I'd appreciate if someone knows what I'm talking about and lets me know. ___________ [1] https://en.wikipedia.org/wiki/Dartmouth_workshop https://en.wikipedia.org/wiki/Dartmouth_workshop [2] https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter
  - moffkalast 3y ago
    We're not at the beginning, but we're not close to the end either. The fact that organic brains exist proves the baseline of what's fundamentally possible and at very low power levels too if optimized well. We haven't reached that yet, and it's impossible to say what the upper limit is. LLMs might not get there as an end-to-end solution but they'll will certainly be a significant part of one.
  - dri_ft 3y ago
    >a planet where winter came in periods too long for the natives of the planet to remember Not Asimov's Nightfall?
    - YeGoblynQueenne 3y ago
      Nope. Winter, not night-time :)
  - grumbel 3y ago
    BingChat says it's Helliconia[1] by Brian W. Aldiss. Nightfall and the movie Pitch Black also go in that direction. [1] https://en.wikipedia.org/wiki/Helliconia https://en.wikipedia.org/wiki/Helliconia
- rootusrootus 3y ago
  I don't know, maybe this is the peak of this kind of AI. It seems like a lot of what drives this boom in LLMs isn't fundamental improvements in the technology, but rather the easy availability of enough computing power to do them. It's plausible that we're going to hit a wall abruptly and that the next step towards true AI will be down a different path.
Solvency 3y ago
The prevailing commentary on Reddit is that GPT4 has been getting progressively nerfed for weeks.
- meghan_rain 3y ago
  [flagged]
  - orangepurple 3y ago
    Tell us more
    - meghan_rain 3y ago
      The more a model is lobotomized to fit a certain narrative ("don't return jokes about women"), the less it performs on normal/safe tasks as well ("summarize this article").
    - rootusrootus 3y ago
      Actually, please don't.
      - meghan_rain 3y ago
        [flagged]
  - SomethingNew2 3y ago
    [flagged]
- eurekin 3y ago
  I use it only for sketching out some programming ideas in python. It takes a lot of back and forth to get anything useful and a lot of times, I have to workaround: > However, these approaches would be a lot more complex and beyond the scope of this assistant's capabilities Which seems very strange
- moffkalast 3y ago
ThunderBee 3y ago
They nerfed the model’s capabilities and made it so that it won’t do many of the interesting things it would at launch. How is anyone shocked?
- basisword 3y ago
  What can it not do anymore? I haven’t noticed any change in my use cases (mostly simple code related questions).
- root_axis 3y ago
  This isn't true.
sourcecodeplz 3y ago
I understand GPT-4 is still as powerful and not nerfed if used via API.
javier123454321 3y ago
I tried, and might be doing it wrong, but it's not helpful for me to use chatGPT. It made almost nothing faster or more convenient, except for maybe making a regex every month or so that I need one. So I end up going to that page about once a month (just checked, and effectively once in June, 4 conversations in May (which were all in the same day), twice in April). Also reviewing those conversations, I did not get a 'useful' response in any of those instances that were not regex related.
TheCaptain4815 3y ago
Interestingly enough, BARD has been gaining new features and functionality. While GPT4 is still better than BARD from my usage, I'd guess the majority of Google searches are quite simple in nature and don't require any type of 'AGI Spark' giving Google a huge benefit. I've been in Googles SGE beta for a few weeks and it's a nice experience. Nonetheless, the winner of all this will be who can 'solve' the hallucination problem. Any startup that comes out with that will become the next Google.
- CuriouslyC 3y ago
  Solving the hallucination problem isn't an engineering challenge, it's a data challenge. Just take the most common non-filler token sequences and annotate them with accuracy scores, use those to train an accuracy predictor, then fine tune the LLM using the accuracy predictor to bias the gradient towards accurate statements. You'd probably have to spend a lot of money (or get a lot of volunteer time) to get it done, so it's not a startup thing, but it would make a massive difference.
rootusrootus 3y ago
I dropped my ChatGPT plus subscription as of the end of July, I'm just using a local client with API access. It'll be much cheaper than $20/month for something that's still largely a toy for me. Unless OpenAI puts a minimum monthly fee on the API now that GPT4 will be available to everyone. I also got into local LLMs and for my purposes they're pretty great. It does help to have an M2 MAX with 96GB of RAM, but even the small models have improved tremendously.
- drapado 3y ago
  Which client are you using? Looking for recommendations
  - rootusrootus 3y ago
    I use MacGPT. Obviously only a recommendation if you're using a Mac, but I am happy with it so far.
- eurekin 3y ago
  Please explain more. I toyed around with few initial Llama derived ones and they didn't seem very useful. I was interested most using them as some for of a coding assitant. Which models work ok for you and for what usages?
  - CuriouslyC 3y ago
    Go to https://www.reddit.com/r/LocalLLaMA/ https://www.reddit.com/r/LocalLLaMA/ and get a 30b uncensored quantized variant with instruction tuning. They work pretty well, pretty close to gpt-3.5 in most domains.
- jon-wood 3y ago
  I'm doing much the same, I've got a tiny Python script that I use to interact with GPT-4 to deal with work related programming questions. For my use case its even better than ChatGPT because I can prime it with a default prompt asking it to be succinct, answer only with code if the answer is some code (rather than providing commentary alongside), and let it know anything I'm asking is in the context of Python running on Linux. With that prompt answers that would previously have been several hundred words of SEO-optimised mush become 5 lines of code I can read to get the idea, and if I need further explanation I can ask for it.
jrm4 3y ago
Good. Now that this stuff is out there, it's too powerful to be locked behind a small number (or perhaps any number of) companies. Free/Open Source here isn't merely a nice ideal, it's probably the only safe way to possibly handle all of this, since it ain't stopping.
atleastoptimal 3y ago
I feel that they've added a classifier model, where it will determine whether a satisfactory answer to a prompt will require the full model or a smaller one. This leads to the inconsistent quality of outputs: some are really from a worse model that the system believes it can get away with and still be "GPT-4" quality. When it takes a long time to get an answer from GPT-4, it seems better, but worse when it's unexpectedly fast.
- nojs 3y ago
  I think this is very likely. It seems like OpenAI is throwing everything at trying to get GPT4 to scale, and this is one of the things they’re experimenting with.
jona-f 3y ago
What a waste of words. The author has no idea of anything. All he does is throw the expression "peak ai" out, thinking he's smart with his buzzwords. Ai is still at the beginning.
Pxtl 3y ago
Is there any reason to use ChatGPT over Bing Chat? I haven't gotten too deep into the more extreme uses of this stuff, just asking occasional questions where google provides irrelevant results.
- Tostino 3y ago
  Bing has been so thoroughly labotamized compared to launch it's not even worth using in my opinion. ChatGPT4 looks like a Rhoads scholar in comparison, even with its own nerfs.
turnsout 3y ago
I don't think it has anything to do with being "nerfed." It's as simple as this: the novelty has worn off. The people who don't have a direct use for it just got bored of asking it silly questions.
- uLogMicheal 3y ago
  I cancelled specifically because it was nerfed and will resume if they take a new path towards openness and transparency.
- colordrops 3y ago
  It is seriously nerfed though. It seems like it won't respond to half of my questions these days. It won't even role play without trepidation at this point. After a few requests it reveals itself to be an AI, perhaps due to OpenAI being worried about people using it for impersonation. It barely responds when asked about people, even public figures. It's overly cautious. I assume we haven't seen the end of it. E.g. at some point they will not allow it to say anything when it comes to medical topics for instance. It's going to be so narrow that it's only good for writing business summaries or some shit. Which I suppose is what they want - people to use it as an API at large corps for business purposes.
  - uLogMicheal 3y ago
    It's also programming tasks. I was working on very complex compute shader work and noticed it more frequently would mention that I should find an expert for certain tasks, or that it can only provide basic examples/frameworks, or it would spit out a bunch of irrelevant junk code not related to the task, leaving placeholders of "/..." where it actually matters. When ChatGPT4 first launched, the system had much less friction for accomplishing the complex. Now, it seems to get lost in the confusion of trying to find reasons NOT to output what I requested.
  - elorant 3y ago
    Large corps will never upload their documents on the API. ChatGPT needs a standalone black box if it's gonna penetrate the corporate world.
CharlieDigital 3y ago
YMMV, but outside of very specialized use cases, it's just not all that useful to the extent that I'd default to using it. I'm building a travel planning app and partner and I toyed around with ChatGPT for generation of itineraries and both agreed that the output was generally OK if only using it for identifying destinations, but otherwise mediocre and not at all how a human would plan travel. It tends to be repetitive in how it picks places for a given destination. Often times, it would hallucinate especially when given constraints like distance (because you'd probably want to get a meal near a destination). There are dozens of startups built around this and even one backed by YC. For coding, it's useful to generate small utility functions, but in many cases, I could have written the utility function or found a snippet on SO instead in roughly equivalent time. Would love to know how HN crowd is finding consistent utility out of the service.
- spaceman_2020 3y ago
  It’s genuinely become a pretty mediocre tool for most things. The text it generates is so formulaic that anyone can spot its AI. It defaults to shallow coding answers. It refuses to answer anything medical or legal without some hardcore prompt engineering. Either its being dumbed down, or OpenAI is genuinely struggling to keep it consistently good. If its the former, it just scares me that a bunch of people have access to a super powerful tool that they’ve intentionally decided to keep from the public. Way too much power in the hands of a few.
- GenerocUsername 3y ago
  That's a really bad usecase for a LLM. A better use would be to suggest activities or destinations based on qualities of those places and a users interests. It can easily generate rough matches, but you would not want to delve into specific mileage based computations.
  - CharlieDigital 3y ago
    > A better use would be to suggest activities or destinations based on qualities of those places and a users interests. For sure, but as spaceman mentions, after a few passes, you can see how formulaic it is. > ...but you would not want to delve into specific mileage based computations. Agree; but it was our own curiosity to see if it could understand geo-spatial locality for this use case. It does have some sense of it likely based on how destinations tend to cluster in the underlying sources, but will obviously include hallucinations in the response as well.
cma 3y ago
Feels like the web version of gpt-4 got worse for coding problems after launch. They only confirmed the API version didn't change.
swader999 3y ago
Not as much homework in the summertime. I do notice it's getting less useful. Find myself going back to Google occasionally.
- bogwog 3y ago
  > Find myself going back to Google occasionally. So ChatGPT is your primary way of finding information?
  - bitsinthesky 3y ago
    Theres a wide range of topics that, as long as you have adequate base knowledge, chatgpt is much better than google. Clean, to the point, tailored to what you seek. Not mass produced filler trash to grab clicks, as counter point.
    - swader999 3y ago
      Dinner recipes really make your point. I just want ingredients, temps and cook times and a quick set of steps to follow. Or, here's a list of ingredients, what can I make? Smashing five pages of text into my eyeballs to get to it is less than optimal.
  - swader999 3y ago
    Yeah I think it's evolved that way. I created a weight program for my son in a few minutes yesterday with it. Was trying to figure out how to do Auth0 with cypress though and I ended up using Google to the cypress and Auth0 docs for that. Most of what I go after is how to do stuff, not facts or data. Recipes etc, chat gpt is fairly useful for that. But less so as time goes by it seems.
nickelcitymario 3y ago
I pity the fool who thinks we've reached "peak AI". We've barely gotten started.
Havoc 3y ago
A but of a hype wave was to be expected I think, so wouldn’t read too much into this
nmpennypacker 3y ago
I use it only for cooking, but for cooking I absolutely love it. It generates surprisingly good recipes, and I don't have to read a 5000 word story on how someone found and fell in love with the turnip to get them.
Roark66 3y ago
I toyed with an idea of actually paying for a subscription, but I'm glad I didn't buy it. It seems other than "write me a nicely worded email about" I almost always use it for edge cases where it hallucinates horribly. So I thought, ok, it's not good enough for any real programming (unless you're writing boilerplate/very basic stuff) so how about I use it to explain to myself certain concepts I find hard to understand (with opportunity to ask questions) and to explore some software capabilities. For example a question like "which of the 4 ML frameworks pytorch, tensorflow, onnx, openvino has out of the box functionality to run inference on models partially in RAM partially on disk"? It got it horribly wrong. Another one, about best profiling tools... Got it wrong too... Do I think those LLMs are a great tool for domain specific knowledge after fine tuning? Sure they are. But a general purpose assistants they aren't.
pc86 3y ago
I'll be honest I'm having a hard time figuring out why I'm paying $20/mo for a service that rate limits me aggressively, seems to have me last on the list for beta features (I think I got plugins 2 days before they were publicly available), and has a non-trivial chance of being under heavy load and borderline unusable whenever I actually go to use it.
TechRemarker 3y ago
It's such an amazing tool, but I others have quickly probably learned it's many limitations in regards to accuracy, where so often it provides a detailed response that seems highly plausible but in my experience so many times can be entirely wrong. For some questions, you are able to test the answer and discover it's wrong and then find a better way to ask to get the answer. Or other type of questions you could tell it's wrong or not as desired simply by looking. As depending on the genhre or how well versed you are with the subject matter plays a big roll. Making it feel less and less useful, since you continue to worry about it's accuracy since it almost never seems to say I don't know but rather always produces a response. So can see how general use would be dropping but also how specific uses remain notably how companies are probably all working on more much more niche integrations that will be able to be more accurate and helpful than using just as an ask anything machine. Looking forward to the future.
koochi10 3y ago
This article is a little bit of a red hearing. OpenAI is not apple, in the sense that they are not great at building user facing products. They are great at building the world's best AI models. They've known this since the inception of the GPT models, the only way you could have accessed these models is via API. Later last year, we saw the release of GPT's text-davinci-003, and in an attempt to showcase this new model to the research community, they launched ChatGPT. I think what we are seeing now is that chatGPT is best when it is close to existing applications. For example what 14 year old is using the chatGPT app vs the Snapchat AI Chat which uses the API internally. The recent drop in usage could likely be attributed to such preferential shifts, further compounded by the timing of school holidays.
- uLogMicheal 3y ago
  Why is it so hard to believe that they have lessened the capability of the chat model? If we had transparency true to the name, we would be able to confirm or deny differences. Now we are stuck in a state of debate / confusion on this. Even if the capability is returned, OpenAI still needs to overcome the grudge they are creating with users in regards to openness. Many may be waiting to jump ship to more open models. Elites gaining access to the best models while everyone else gets the censored/delayed rollout in the name of safety needs to stop. OpenAI should rebrand or return to core values. Sure they contribute to open source, but do they contribute their best to open source as originally intended?
  - koochi10 3y ago
    What proof do we have that they "lessened the capability", so far I've only seen rumors on Hackernews. Even if they were malicious what benefit does Openai get from lessening the model to its user's only to give it to "Elites"? This sounds like a conspiracy theory to me
    - uLogMicheal 3y ago
      Your response does two things -- Dismisses the countless examples given, from me in this thread/my comment history, and many other people in many threads on this website and Reddit. Dismisses the pretense that certain people get access to unfiltered models under the guise of conspiracy. In the sparks of AGI paper from Microsoft, the researcher mentions the differences in private/test models versus the ones prepped for consumers. If you want a hard to ignore visual example, just look at the unicorn they drew for that paper and then look at https://gpt-unicorn.adamkdean.co.uk/ https://gpt-unicorn.adamkdean.co.uk/. I hope you can add to these discussions rather than primarily be dismissive. This is not conspiracy, we do not know the intention behind the changes and I am not speculating on the intentions of the actors. Sparks of AGI: https://arxiv.org/pdf/2303.12712.pdf https://arxiv.org/pdf/2303.12712.pdf
aCoreyJ 3y ago
I wonder if this includes if you are just using bing, phind or copilot

48 points by noufalibrahim 3y ago

I have seen it posted before, SimilarWeb numbers are questionable. If OpenAI can publish user stats, I believe the claim as long as they are open about their methodology.

damnesian 3y ago

>Have we hit peak AI? AI is at the same point cinema was at when pioneers of the genre showed short films in "salons" designed to frighten people with simple tricks. This is hardly the peak. We're just at the beginning.

orwin 3y ago
AI development is different than most CS fields, as you need a solid mathematical base to build a "new discovery", and a lot of computation to use it efficiently. I'm 100% certain it's not the AI peak, but i'm also confident that we are close to the local maxima, at least for transformer architecture. Maybe the following improvement will be in chaining architecture, or a brand new technique, or maybe progress will be made in parallel fields (even smaller processors!). Charitably, i'll decide that the question meant "Have we hit peak GPT" and in this case, this is a legitimate question.
- unshavedyak 3y ago
  I'm most interested in refined training sets, refined compression, etc. Where do you suspect the cap is there? I say that because i find GPT4 extremely useful. Even if GPT4 (in it's hypothetical un-nerfed state some would argue)* is the peak LLM i think we have a ton of room to grow even within that peak. Locally running larger context windows, multiple domain experts in parallel and hooked up to a local repository of available IO actions would give a very significant "advancement" within the same conceptual "peak LLM". Now will we be able to get there on consumer hardware? Not idea. However i don't think i need more than GPT4 to find the whole technology extremely useful. Moderately cheap to run large context windows[1] entirely locally is the "next big thing" for me. I get a fair bit of value from GPT4 and Phind. The technology has sold me already. [1]: I should amend, i'm speaking loosely about context windows. What i mean is the ability to feed it refined datasets locally to have it's "local knowledge" change, which may be less about context windows and more about refined training. I'll leave that to the experts to debate, i am assuredly nothing close.
  - orwin 3y ago
    > I say that because i find GPT4 extremely useful. Even if GPT4 (in it's hypothetical un-nerfed state some would argue)* is the peak LLM i think we have a ton of room to grow even within that peak. Yes. sorry my point wasn't well formulated (i'm not an expert either: i've trained LLMs on receipts in 2016, and quickly decided i hated that). I'll try doing it better by dividing the two idea i put in that post: - i think that GPT4 is probably close the peak generalist GPT, meaning that maybe Bard can surpass it (or a chatGPT4.5 or something) but the improvement won't be stellar. - If future improvement won't come from the transformer architecture itself (i absolutely can be wrong on that), it'll come from what we interface with it. Absolutely context windows will probably a great way to have "specialists" transformers, but alos linking it with different "kind" of AI might push it too (i've read a post from Wolfram a while ago that sold me on interlinking different AI models/languages/state machine). I'm sure specialist or even large users haves tons of ideas. I got some value from GPT3.5, enough for me to buy access to GPT4. with a caveat: i'm pretty sure that for work-related questions i lost a bit more time than i "gained" overall. because my mastery of the technologies we use and very specific needs that GPT cannot know. But in the last three weeks or so, i'm pretty sure it was overall a net benefit at work, as i know better when i should use it.
- CuriouslyC 3y ago
  I don't think we'll need to completely replace the transformer, we just need better, more annotated data sets and a really good solution to large context windows. Data set annotation has a huge impact on model output, you can see this for yourself by comparing SDXL to Stable Diffusion 1.5, the model is larger but the real juice comes from using aesthetics scores for images during training to bias output towards things that look good - compare it to other large models like DALL-E or Imagen and you'll see parameter size is not the most important thing. With larger context windows, we can use traditional search algorithms to cherry pick an educational context for the LLM for few shot learning. If that's paired with more, better and annotated data, that will take us quite far.
YeGoblynQueenne 3y ago
We're not at the beginning. AI research started in 1956 with the Dartmouth workshop [1]. We've already had several cycles of boom and bust and at least two AI winters already [2]. Btw, there was a sci-fi novel where the plot took place on a planet where winter came in periods too long for the natives of the planet to remember (they had short lifespans, or they had limited memories, I er, why, I can't remember!). I don't remember the title of the novel or the author and I've been trying to find it. I thought it was something by Ursula K. Le Guin, but it doesn't sound like it. And no, it's not Game of Thrones. I think GoT took the idea of long-coming winters from the earlier novel, or it's just a coincidence (or cryptomnesia). Anyway I'd appreciate if someone knows what I'm talking about and lets me know. ___________ [1] https://en.wikipedia.org/wiki/Dartmouth_workshop https://en.wikipedia.org/wiki/Dartmouth_workshop [2] https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter
- moffkalast 3y ago
  We're not at the beginning, but we're not close to the end either. The fact that organic brains exist proves the baseline of what's fundamentally possible and at very low power levels too if optimized well. We haven't reached that yet, and it's impossible to say what the upper limit is. LLMs might not get there as an end-to-end solution but they'll will certainly be a significant part of one.
- dri_ft 3y ago
  >a planet where winter came in periods too long for the natives of the planet to remember Not Asimov's Nightfall?
  - YeGoblynQueenne 3y ago
    Nope. Winter, not night-time :)
- grumbel 3y ago
  BingChat says it's Helliconia[1] by Brian W. Aldiss. Nightfall and the movie Pitch Black also go in that direction. [1] https://en.wikipedia.org/wiki/Helliconia https://en.wikipedia.org/wiki/Helliconia
rootusrootus 3y ago
I don't know, maybe this is the peak of this kind of AI. It seems like a lot of what drives this boom in LLMs isn't fundamental improvements in the technology, but rather the easy availability of enough computing power to do them. It's plausible that we're going to hit a wall abruptly and that the next step towards true AI will be down a different path.

Solvency 3y ago

The prevailing commentary on Reddit is that GPT4 has been getting progressively nerfed for weeks.

meghan_rain 3y ago
[flagged]
- orangepurple 3y ago
  Tell us more
  - meghan_rain 3y ago
    The more a model is lobotomized to fit a certain narrative ("don't return jokes about women"), the less it performs on normal/safe tasks as well ("summarize this article").
  - rootusrootus 3y ago
    Actually, please don't.
    - meghan_rain 3y ago
      [flagged]
- SomethingNew2 3y ago
  [flagged]
eurekin 3y ago
I use it only for sketching out some programming ideas in python. It takes a lot of back and forth to get anything useful and a lot of times, I have to workaround: > However, these approaches would be a lot more complex and beyond the scope of this assistant's capabilities Which seems very strange
moffkalast 3y ago

ThunderBee 3y ago

They nerfed the model’s capabilities and made it so that it won’t do many of the interesting things it would at launch. How is anyone shocked?

basisword 3y ago
What can it not do anymore? I haven’t noticed any change in my use cases (mostly simple code related questions).
root_axis 3y ago
This isn't true.

sourcecodeplz 3y ago

I understand GPT-4 is still as powerful and not nerfed if used via API.

javier123454321 3y ago

I tried, and might be doing it wrong, but it's not helpful for me to use chatGPT. It made almost nothing faster or more convenient, except for maybe making a regex every month or so that I need one. So I end up going to that page about once a month (just checked, and effectively once in June, 4 conversations in May (which were all in the same day), twice in April). Also reviewing those conversations, I did not get a 'useful' response in any of those instances that were not regex related.

TheCaptain4815 3y ago

Interestingly enough, BARD has been gaining new features and functionality. While GPT4 is still better than BARD from my usage, I'd guess the majority of Google searches are quite simple in nature and don't require any type of 'AGI Spark' giving Google a huge benefit. I've been in Googles SGE beta for a few weeks and it's a nice experience. Nonetheless, the winner of all this will be who can 'solve' the hallucination problem. Any startup that comes out with that will become the next Google.

CuriouslyC 3y ago
Solving the hallucination problem isn't an engineering challenge, it's a data challenge. Just take the most common non-filler token sequences and annotate them with accuracy scores, use those to train an accuracy predictor, then fine tune the LLM using the accuracy predictor to bias the gradient towards accurate statements. You'd probably have to spend a lot of money (or get a lot of volunteer time) to get it done, so it's not a startup thing, but it would make a massive difference.

rootusrootus 3y ago

I dropped my ChatGPT plus subscription as of the end of July, I'm just using a local client with API access. It'll be much cheaper than $20/month for something that's still largely a toy for me. Unless OpenAI puts a minimum monthly fee on the API now that GPT4 will be available to everyone. I also got into local LLMs and for my purposes they're pretty great. It does help to have an M2 MAX with 96GB of RAM, but even the small models have improved tremendously.

drapado 3y ago
Which client are you using? Looking for recommendations
- rootusrootus 3y ago
  I use MacGPT. Obviously only a recommendation if you're using a Mac, but I am happy with it so far.
eurekin 3y ago
Please explain more. I toyed around with few initial Llama derived ones and they didn't seem very useful. I was interested most using them as some for of a coding assitant. Which models work ok for you and for what usages?
- CuriouslyC 3y ago
  Go to https://www.reddit.com/r/LocalLLaMA/ https://www.reddit.com/r/LocalLLaMA/ and get a 30b uncensored quantized variant with instruction tuning. They work pretty well, pretty close to gpt-3.5 in most domains.
jon-wood 3y ago
I'm doing much the same, I've got a tiny Python script that I use to interact with GPT-4 to deal with work related programming questions. For my use case its even better than ChatGPT because I can prime it with a default prompt asking it to be succinct, answer only with code if the answer is some code (rather than providing commentary alongside), and let it know anything I'm asking is in the context of Python running on Linux. With that prompt answers that would previously have been several hundred words of SEO-optimised mush become 5 lines of code I can read to get the idea, and if I need further explanation I can ask for it.

jrm4 3y ago

Good. Now that this stuff is out there, it's too powerful to be locked behind a small number (or perhaps any number of) companies. Free/Open Source here isn't merely a nice ideal, it's probably the only safe way to possibly handle all of this, since it ain't stopping.

atleastoptimal 3y ago

I feel that they've added a classifier model, where it will determine whether a satisfactory answer to a prompt will require the full model or a smaller one. This leads to the inconsistent quality of outputs: some are really from a worse model that the system believes it can get away with and still be "GPT-4" quality. When it takes a long time to get an answer from GPT-4, it seems better, but worse when it's unexpectedly fast.

nojs 3y ago
I think this is very likely. It seems like OpenAI is throwing everything at trying to get GPT4 to scale, and this is one of the things they’re experimenting with.

jona-f 3y ago

What a waste of words. The author has no idea of anything. All he does is throw the expression "peak ai" out, thinking he's smart with his buzzwords. Ai is still at the beginning.

Pxtl 3y ago

Is there any reason to use ChatGPT over Bing Chat? I haven't gotten too deep into the more extreme uses of this stuff, just asking occasional questions where google provides irrelevant results.

Tostino 3y ago
Bing has been so thoroughly labotamized compared to launch it's not even worth using in my opinion. ChatGPT4 looks like a Rhoads scholar in comparison, even with its own nerfs.

turnsout 3y ago

I don't think it has anything to do with being "nerfed." It's as simple as this: the novelty has worn off. The people who don't have a direct use for it just got bored of asking it silly questions.

uLogMicheal 3y ago
I cancelled specifically because it was nerfed and will resume if they take a new path towards openness and transparency.
colordrops 3y ago
It is seriously nerfed though. It seems like it won't respond to half of my questions these days. It won't even role play without trepidation at this point. After a few requests it reveals itself to be an AI, perhaps due to OpenAI being worried about people using it for impersonation. It barely responds when asked about people, even public figures. It's overly cautious. I assume we haven't seen the end of it. E.g. at some point they will not allow it to say anything when it comes to medical topics for instance. It's going to be so narrow that it's only good for writing business summaries or some shit. Which I suppose is what they want - people to use it as an API at large corps for business purposes.
- uLogMicheal 3y ago
  It's also programming tasks. I was working on very complex compute shader work and noticed it more frequently would mention that I should find an expert for certain tasks, or that it can only provide basic examples/frameworks, or it would spit out a bunch of irrelevant junk code not related to the task, leaving placeholders of "/..." where it actually matters. When ChatGPT4 first launched, the system had much less friction for accomplishing the complex. Now, it seems to get lost in the confusion of trying to find reasons NOT to output what I requested.
- elorant 3y ago
  Large corps will never upload their documents on the API. ChatGPT needs a standalone black box if it's gonna penetrate the corporate world.

CharlieDigital 3y ago

YMMV, but outside of very specialized use cases, it's just not all that useful to the extent that I'd default to using it. I'm building a travel planning app and partner and I toyed around with ChatGPT for generation of itineraries and both agreed that the output was generally OK if only using it for identifying destinations, but otherwise mediocre and not at all how a human would plan travel. It tends to be repetitive in how it picks places for a given destination. Often times, it would hallucinate especially when given constraints like distance (because you'd probably want to get a meal near a destination). There are dozens of startups built around this and even one backed by YC. For coding, it's useful to generate small utility functions, but in many cases, I could have written the utility function or found a snippet on SO instead in roughly equivalent time. Would love to know how HN crowd is finding consistent utility out of the service.

spaceman_2020 3y ago
It’s genuinely become a pretty mediocre tool for most things. The text it generates is so formulaic that anyone can spot its AI. It defaults to shallow coding answers. It refuses to answer anything medical or legal without some hardcore prompt engineering. Either its being dumbed down, or OpenAI is genuinely struggling to keep it consistently good. If its the former, it just scares me that a bunch of people have access to a super powerful tool that they’ve intentionally decided to keep from the public. Way too much power in the hands of a few.
GenerocUsername 3y ago
That's a really bad usecase for a LLM. A better use would be to suggest activities or destinations based on qualities of those places and a users interests. It can easily generate rough matches, but you would not want to delve into specific mileage based computations.
- CharlieDigital 3y ago
  > A better use would be to suggest activities or destinations based on qualities of those places and a users interests. For sure, but as spaceman mentions, after a few passes, you can see how formulaic it is. > ...but you would not want to delve into specific mileage based computations. Agree; but it was our own curiosity to see if it could understand geo-spatial locality for this use case. It does have some sense of it likely based on how destinations tend to cluster in the underlying sources, but will obviously include hallucinations in the response as well.

cma 3y ago

Feels like the web version of gpt-4 got worse for coding problems after launch. They only confirmed the API version didn't change.

swader999 3y ago

Not as much homework in the summertime. I do notice it's getting less useful. Find myself going back to Google occasionally.

bogwog 3y ago
> Find myself going back to Google occasionally. So ChatGPT is your primary way of finding information?
- bitsinthesky 3y ago
  Theres a wide range of topics that, as long as you have adequate base knowledge, chatgpt is much better than google. Clean, to the point, tailored to what you seek. Not mass produced filler trash to grab clicks, as counter point.
  - swader999 3y ago
    Dinner recipes really make your point. I just want ingredients, temps and cook times and a quick set of steps to follow. Or, here's a list of ingredients, what can I make? Smashing five pages of text into my eyeballs to get to it is less than optimal.
- swader999 3y ago
  Yeah I think it's evolved that way. I created a weight program for my son in a few minutes yesterday with it. Was trying to figure out how to do Auth0 with cypress though and I ended up using Google to the cypress and Auth0 docs for that. Most of what I go after is how to do stuff, not facts or data. Recipes etc, chat gpt is fairly useful for that. But less so as time goes by it seems.

nickelcitymario 3y ago

I pity the fool who thinks we've reached "peak AI". We've barely gotten started.

Havoc 3y ago

A but of a hype wave was to be expected I think, so wouldn’t read too much into this

nmpennypacker 3y ago

I use it only for cooking, but for cooking I absolutely love it. It generates surprisingly good recipes, and I don't have to read a 5000 word story on how someone found and fell in love with the turnip to get them.

Roark66 3y ago

I toyed with an idea of actually paying for a subscription, but I'm glad I didn't buy it. It seems other than "write me a nicely worded email about" I almost always use it for edge cases where it hallucinates horribly. So I thought, ok, it's not good enough for any real programming (unless you're writing boilerplate/very basic stuff) so how about I use it to explain to myself certain concepts I find hard to understand (with opportunity to ask questions) and to explore some software capabilities. For example a question like "which of the 4 ML frameworks pytorch, tensorflow, onnx, openvino has out of the box functionality to run inference on models partially in RAM partially on disk"? It got it horribly wrong. Another one, about best profiling tools... Got it wrong too... Do I think those LLMs are a great tool for domain specific knowledge after fine tuning? Sure they are. But a general purpose assistants they aren't.

pc86 3y ago

I'll be honest I'm having a hard time figuring out why I'm paying $20/mo for a service that rate limits me aggressively, seems to have me last on the list for beta features (I think I got plugins 2 days before they were publicly available), and has a non-trivial chance of being under heavy load and borderline unusable whenever I actually go to use it.

TechRemarker 3y ago

It's such an amazing tool, but I others have quickly probably learned it's many limitations in regards to accuracy, where so often it provides a detailed response that seems highly plausible but in my experience so many times can be entirely wrong. For some questions, you are able to test the answer and discover it's wrong and then find a better way to ask to get the answer. Or other type of questions you could tell it's wrong or not as desired simply by looking. As depending on the genhre or how well versed you are with the subject matter plays a big roll. Making it feel less and less useful, since you continue to worry about it's accuracy since it almost never seems to say I don't know but rather always produces a response. So can see how general use would be dropping but also how specific uses remain notably how companies are probably all working on more much more niche integrations that will be able to be more accurate and helpful than using just as an ask anything machine. Looking forward to the future.

koochi10 3y ago

This article is a little bit of a red hearing. OpenAI is not apple, in the sense that they are not great at building user facing products. They are great at building the world's best AI models. They've known this since the inception of the GPT models, the only way you could have accessed these models is via API. Later last year, we saw the release of GPT's text-davinci-003, and in an attempt to showcase this new model to the research community, they launched ChatGPT. I think what we are seeing now is that chatGPT is best when it is close to existing applications. For example what 14 year old is using the chatGPT app vs the Snapchat AI Chat which uses the API internally. The recent drop in usage could likely be attributed to such preferential shifts, further compounded by the timing of school holidays.

uLogMicheal 3y ago
Why is it so hard to believe that they have lessened the capability of the chat model? If we had transparency true to the name, we would be able to confirm or deny differences. Now we are stuck in a state of debate / confusion on this. Even if the capability is returned, OpenAI still needs to overcome the grudge they are creating with users in regards to openness. Many may be waiting to jump ship to more open models. Elites gaining access to the best models while everyone else gets the censored/delayed rollout in the name of safety needs to stop. OpenAI should rebrand or return to core values. Sure they contribute to open source, but do they contribute their best to open source as originally intended?
- koochi10 3y ago
  What proof do we have that they "lessened the capability", so far I've only seen rumors on Hackernews. Even if they were malicious what benefit does Openai get from lessening the model to its user's only to give it to "Elites"? This sounds like a conspiracy theory to me
  - uLogMicheal 3y ago
    Your response does two things -- Dismisses the countless examples given, from me in this thread/my comment history, and many other people in many threads on this website and Reddit. Dismisses the pretense that certain people get access to unfiltered models under the guise of conspiracy. In the sparks of AGI paper from Microsoft, the researcher mentions the differences in private/test models versus the ones prepped for consumers. If you want a hard to ignore visual example, just look at the unicorn they drew for that paper and then look at https://gpt-unicorn.adamkdean.co.uk/ https://gpt-unicorn.adamkdean.co.uk/. I hope you can add to these discussions rather than primarily be dismissive. This is not conspiracy, we do not know the intention behind the changes and I am not speculating on the intentions of the actors. Sparks of AGI: https://arxiv.org/pdf/2303.12712.pdf https://arxiv.org/pdf/2303.12712.pdf

aCoreyJ 3y ago

I wonder if this includes if you are just using bing, phind or copilot