4 ms·

We need a lot more of this kind of in depth analysis. Right now the cheering on of AI is overwhelming. Criticism is often suppressed, both on the vendor side wh

We need a lot more of this kind of in depth analysis. Right now the cheering on of AI is overwhelming. Criticism is often suppressed, both on the vendor side who just want to sell, and on the client side who have very strong FOMO. I work for the client side and this bothers me a lot. It's very hard to get a true honest value analysis done with all the sales influence and office politics going on.

LudwigNagasena 2y ago
I would expect an in-depth analysis to at least include a comparison of various prompts.
- whiplash451 2y ago
  Indeed. And more models as well. That being said, these initial results are not reassuring.
- Spivak 2y ago
  I have actually made (what I think to be) a working summarizer at $dayjob and it took a lot more hand-holding to get results than I initially expected. A straight summary wasn't very good, the "summary of summaries" approach as implemented by LangChain was garbage and didn't produce a summary at all, even a wrong one. The algorithm that actually worked: 1. Take the documents, chunk them up on paragraph then sentence then word boundaries using spaCy. 2. Generate embeddings for each chunk and cluster them using the silhouette score to estimate the number of clusters. 3. Take the top 3 documents closest to the centroid of each cluster, expand the context before and after so it's 9 chunks in 3 groups. 4. For each cluster ask the LLM to extract the key points as direct quotes from the document. 5. Take those quotes and match them up to the real document to make sure it didn't just make stuff up. 6. Then put all the quotes together and ask the LLM not to summarize, but to write the information presented in paragraph form. 7. Then because LLMs just can't seem to shut up about their answer make them return JSON {"summary": "", "commentary": ""} and discard the commentary. The LLM performs much better (to human reviewers) at keyphrase extraction than TextRank so I think there's genuinely some value there and obviously nothing else can really compose english like these models but I think we perhaps expect too much out of the "raw" model.
  - __loam 2y ago
    It's cool to hear about something substantial that isn't just an api call to the "God" machine. This sounds like a pretty sophisticated and well considered approach, but it does call in to question whether a system that needs to be used as a subsystem in a larger one to be reliable is worth the amount of money being invested in making them right now.
- moffkalast 2y ago
  Part of the problem IMO might be that OP is relying on OAI and Google's systems to dump data from websites and pdfs into the context and hoping it's correct and formatted properly. It probably is, but the odd random skips could also be explained by copy paste fails. Would be a better comparison if it was just provided in plain text. It's also all long context models getting pages of data, which even for these flagship ones is certainly just RoPE or similar which is a cheap hack but isn't super accurate [0]. 4o is best and still showing haystack benchmark accuracies below 80% and Gemini is just completely blind. That certainly needs fixing up to 100% before we can say for sure that nothing will ever get skipped. [0] https://preview.redd.it/rlgauej7ve4d1.jpeg?width=1086&format=pjpg&auto=webp&s=fa4e288f8f62c75523993aa4a15867d11889d508 https://preview.redd.it/rlgauej7ve4d1.jpeg?width=1086&format...
badgersnake 2y ago
I think most people have realised AI is wildly overhyped at this point.
- OrigamiPastrami 2y ago
  I'm not even sure most people on HN feel that way, although there's plenty of criticism here. Most of the time I use ChatGPT I become genuinely upset within a few minutes because it's so brazenly awful and regularly ignores what I ask it. I see people on here promoting its ability to help them with writing code. I wonder if these are people that just don't know what good code looks like.
  - ta8645 2y ago
    I recognize the hype for what it is, but I find the people calling for "sanity" are overrepresented everywhere; and it seems to me they discount the actual utility of AI way too much. In other words, I think the AI-skepticism is overhyped, just as much as AI itself. For my uses, I find AI to be a much better search/answer engine than Google ever was. It can produce answers with hyperlinks for further reading much better and more efficiently than any other option. I no longer have to read through a bunch of seemingly random Google search results, hoping that my specific question is addressed.
    - lottin 2y ago
      I think it's a matter of personal preference. I much prefer going through a series of search results than reading anything a chatbot has to say about any topic.
  - Bumblonono 2y ago
    Plenty of people understand that LLM is not AI. Nonethless, try looking more left and right. There are really good opensource solutions which can surfe the web for you like stormai, Or you can use anythingllm and give it all your local files. You can write your tips and tricks, start commands, upgrade procedures etc. in Markdown and reference it through your local LLM. I like it for coding specifically for languages or things i write seldomly (i'm not coding every day but did for 15 years). Nonetheless, googles internal code review tool is already suggesting things which are getting accepted by more than 50%. Thats a lot and will only get better. GitHub with Copilot will also just get better every day too. They probably struggled (as the whole industry) with actually getting used to having ML stuff in our ecosystem. Its still relativly new.
chad1n 2y ago
If you complain the the current generation of LLMs isn't as impressive as some people try to prove it is, the shills complain that you used the wrong prompts and maybe you should fine-tune the model. I don't think people should spend much time trying to write a prompt to find a working solution because they could just do that. In the author's case, if he used the summary provided by ChatGPT, I doubt that anyone would have realized it's wrong, but that doesn't mean it's good either. The problem with LLMs is the lack of reliability and coherence. If it gives you a wrong answer and you ask it if it's sure, in most cases it will show another wrong answer and you need to go through multiple of these hops to get something fine.
- apwell23 2y ago
  > If you complain the the current generation of LLMs isn't as impressive as some people try to prove it is, the shills complain that you used the wrong prompts I think the go to is everyone said the same thing about the internet. Look where we are now.
  - Sharlin 2y ago
    Only after the bursting of a very big hype bubble, remember? AI is also different in the sense that it has already gone through several hype cycles, each followed by an "AI winter" of broken dreams. Clearly large DL models are a breakthrough, but the amount of hype and hot air is entirely out of proportion to the actual results.
- rplnt 2y ago
  This "sorry, I was wrong, here's another wrong answer" is basically my only experience with llm. But it also onpy works if I know the answer or the answer I get is really stupid. Luckily the latter is common.
23B1 2y ago
Tech is one of the best places to be a rampant liar because the money is self-reinforcing and everyone who might call you out... is also getting paid by the same lie or liars!
apwell23 2y ago
> Right now the cheering on of AI is overwhelming. I think the tide is turning for sure. Metaverse scam unraveled pretty fast after building for months.
- recursive 2y ago
  Remember NFTs?
  - apwell23 2y ago
    NFT probably was much tamer. There was no trillion dollar companies renaming themselves to metaverse, no gartner predictions of 30 trillion by 2030, ppl spending 1 hr everyday in the metaverse all with no actual definition or care for what metaverse actually was. NFT atleast had clear definition. WTF is metaverse. How the fuck did zuckerberg and nadella get away with metaverse farce. I feel so cynical about tech and world in general.
ninetyninenine 2y ago
>Right now the cheering on of AI is overwhelming It's both, the level of criticism is also overwhelming and delusional. I hear things like, AI will never be as smart as me!, It will never take over my job, Look it got this and this wrong, No chance it can ever be as smart as me! It's always a comparison to their own abilities so I think a lot of it is just an attempt to stay relevant in a world where technology is about to replace all of us. I think by now everyone knows the limitations of current LLMs. The result of this "in depth" analysis is not only expected but very obvious. Nobody is surprised and none of this is suppressed because it's so obvious. What's delusional is the amount of criticism around the performance. The denial that AI is more and more matching human intelligence. Of course it's not there yet, but LLMs made a giant leap and bridged a huge gap. AI is no match for humanity now, that much is obvious. I think the delusion lies in the fact that many people are trying to deny the trendline... the obvious future and trajectory of what progress has been pointing too. Milestones are getting surpassed at a frightening pace. AI is now running circles around the turing test and I can now ride a car with no driver and it's normal in SF. We are here bitching about the fact that AI shortens a text rather then summarizing it without remarking at the fact that you can actually bitch to the AI directly about this fact and demand the AI to stop shortening the text and start summarizing it.
- __loam 2y ago
  The problem with this take is that you're conflating different software systems under the same vague AI umbrella. Waymo is a completely different technology than generative AI. Talking about how over a decade and billions of dollars in R&d in self driving is finally bearing fruit has nothing to do with the performance of LLMs. It is true that LLMs appear to be an exciting technology, but it's also delusional to assume they're following a positive trendline. Performance between GPT-3.5 and GPT-4 was like a 25% improvement that took 2500% more resources to train. It's clear that there's diminishing returns to bigger and bigger models, and the trend we've been seeing in the industry is actually smaller models trained for longer periods in an effort to bring down inference costs while maintaining current performance. More intelligent models may require new techniques and technologies that we don't have yet. I'm sure it will get better but the path to improving isn't as obvious as you're making it sound. Making comparisons to Moore's law is also disengenuous because we're actually running into physical limits on how dense we can make chips due to the size of atoms themselves, so past trends for technological development may not continue to bear out.
stareatgoats 2y ago
> Right now the cheering on of AI is overwhelming Perhaps everywhere else. On HN mostly what I see are these fairly shallow dismissals TBH. It is a natural reaction when your own livelihood is affected, old as tech itself to be sure [0]. Still, it's getting tiresome. A new technological revolution is unfolding, and the people best positioned to lead and keep it in check are largely balking. [0] https://en.wikipedia.org/wiki/Luddite https://en.wikipedia.org/wiki/Luddite
parker-3461 2y ago
I would only consider this as “in depth” if and only if the author provided clearer methodology on the exact model used for this write up. It is still unclear to me whether this is a deficiency in all ChatGPT models or this is just one of the many.

LudwigNagasena 2y ago

I would expect an in-depth analysis to at least include a comparison of various prompts.

whiplash451 2y ago
Indeed. And more models as well. That being said, these initial results are not reassuring.
Spivak 2y ago
I have actually made (what I think to be) a working summarizer at $dayjob and it took a lot more hand-holding to get results than I initially expected. A straight summary wasn't very good, the "summary of summaries" approach as implemented by LangChain was garbage and didn't produce a summary at all, even a wrong one. The algorithm that actually worked: 1. Take the documents, chunk them up on paragraph then sentence then word boundaries using spaCy. 2. Generate embeddings for each chunk and cluster them using the silhouette score to estimate the number of clusters. 3. Take the top 3 documents closest to the centroid of each cluster, expand the context before and after so it's 9 chunks in 3 groups. 4. For each cluster ask the LLM to extract the key points as direct quotes from the document. 5. Take those quotes and match them up to the real document to make sure it didn't just make stuff up. 6. Then put all the quotes together and ask the LLM not to summarize, but to write the information presented in paragraph form. 7. Then because LLMs just can't seem to shut up about their answer make them return JSON {"summary": "", "commentary": ""} and discard the commentary. The LLM performs much better (to human reviewers) at keyphrase extraction than TextRank so I think there's genuinely some value there and obviously nothing else can really compose english like these models but I think we perhaps expect too much out of the "raw" model.
- __loam 2y ago
  It's cool to hear about something substantial that isn't just an api call to the "God" machine. This sounds like a pretty sophisticated and well considered approach, but it does call in to question whether a system that needs to be used as a subsystem in a larger one to be reliable is worth the amount of money being invested in making them right now.
moffkalast 2y ago
Part of the problem IMO might be that OP is relying on OAI and Google's systems to dump data from websites and pdfs into the context and hoping it's correct and formatted properly. It probably is, but the odd random skips could also be explained by copy paste fails. Would be a better comparison if it was just provided in plain text. It's also all long context models getting pages of data, which even for these flagship ones is certainly just RoPE or similar which is a cheap hack but isn't super accurate [0]. 4o is best and still showing haystack benchmark accuracies below 80% and Gemini is just completely blind. That certainly needs fixing up to 100% before we can say for sure that nothing will ever get skipped. [0] https://preview.redd.it/rlgauej7ve4d1.jpeg?width=1086&format=pjpg&auto=webp&s=fa4e288f8f62c75523993aa4a15867d11889d508 https://preview.redd.it/rlgauej7ve4d1.jpeg?width=1086&format...

badgersnake 2y ago

I think most people have realised AI is wildly overhyped at this point.

OrigamiPastrami 2y ago
I'm not even sure most people on HN feel that way, although there's plenty of criticism here. Most of the time I use ChatGPT I become genuinely upset within a few minutes because it's so brazenly awful and regularly ignores what I ask it. I see people on here promoting its ability to help them with writing code. I wonder if these are people that just don't know what good code looks like.
- ta8645 2y ago
  I recognize the hype for what it is, but I find the people calling for "sanity" are overrepresented everywhere; and it seems to me they discount the actual utility of AI way too much. In other words, I think the AI-skepticism is overhyped, just as much as AI itself. For my uses, I find AI to be a much better search/answer engine than Google ever was. It can produce answers with hyperlinks for further reading much better and more efficiently than any other option. I no longer have to read through a bunch of seemingly random Google search results, hoping that my specific question is addressed.
  - lottin 2y ago
    I think it's a matter of personal preference. I much prefer going through a series of search results than reading anything a chatbot has to say about any topic.
- Bumblonono 2y ago
  Plenty of people understand that LLM is not AI. Nonethless, try looking more left and right. There are really good opensource solutions which can surfe the web for you like stormai, Or you can use anythingllm and give it all your local files. You can write your tips and tricks, start commands, upgrade procedures etc. in Markdown and reference it through your local LLM. I like it for coding specifically for languages or things i write seldomly (i'm not coding every day but did for 15 years). Nonetheless, googles internal code review tool is already suggesting things which are getting accepted by more than 50%. Thats a lot and will only get better. GitHub with Copilot will also just get better every day too. They probably struggled (as the whole industry) with actually getting used to having ML stuff in our ecosystem. Its still relativly new.

chad1n 2y ago

If you complain the the current generation of LLMs isn't as impressive as some people try to prove it is, the shills complain that you used the wrong prompts and maybe you should fine-tune the model. I don't think people should spend much time trying to write a prompt to find a working solution because they could just do that. In the author's case, if he used the summary provided by ChatGPT, I doubt that anyone would have realized it's wrong, but that doesn't mean it's good either. The problem with LLMs is the lack of reliability and coherence. If it gives you a wrong answer and you ask it if it's sure, in most cases it will show another wrong answer and you need to go through multiple of these hops to get something fine.

apwell23 2y ago
> If you complain the the current generation of LLMs isn't as impressive as some people try to prove it is, the shills complain that you used the wrong prompts I think the go to is everyone said the same thing about the internet. Look where we are now.
- Sharlin 2y ago
  Only after the bursting of a very big hype bubble, remember? AI is also different in the sense that it has already gone through several hype cycles, each followed by an "AI winter" of broken dreams. Clearly large DL models are a breakthrough, but the amount of hype and hot air is entirely out of proportion to the actual results.
rplnt 2y ago
This "sorry, I was wrong, here's another wrong answer" is basically my only experience with llm. But it also onpy works if I know the answer or the answer I get is really stupid. Luckily the latter is common.

23B1 2y ago

Tech is one of the best places to be a rampant liar because the money is self-reinforcing and everyone who might call you out... is also getting paid by the same lie or liars!

apwell23 2y ago

> Right now the cheering on of AI is overwhelming. I think the tide is turning for sure. Metaverse scam unraveled pretty fast after building for months.

recursive 2y ago
Remember NFTs?
- apwell23 2y ago
  NFT probably was much tamer. There was no trillion dollar companies renaming themselves to metaverse, no gartner predictions of 30 trillion by 2030, ppl spending 1 hr everyday in the metaverse all with no actual definition or care for what metaverse actually was. NFT atleast had clear definition. WTF is metaverse. How the fuck did zuckerberg and nadella get away with metaverse farce. I feel so cynical about tech and world in general.

ninetyninenine 2y ago

>Right now the cheering on of AI is overwhelming It's both, the level of criticism is also overwhelming and delusional. I hear things like, AI will never be as smart as me!, It will never take over my job, Look it got this and this wrong, No chance it can ever be as smart as me! It's always a comparison to their own abilities so I think a lot of it is just an attempt to stay relevant in a world where technology is about to replace all of us. I think by now everyone knows the limitations of current LLMs. The result of this "in depth" analysis is not only expected but very obvious. Nobody is surprised and none of this is suppressed because it's so obvious. What's delusional is the amount of criticism around the performance. The denial that AI is more and more matching human intelligence. Of course it's not there yet, but LLMs made a giant leap and bridged a huge gap. AI is no match for humanity now, that much is obvious. I think the delusion lies in the fact that many people are trying to deny the trendline... the obvious future and trajectory of what progress has been pointing too. Milestones are getting surpassed at a frightening pace. AI is now running circles around the turing test and I can now ride a car with no driver and it's normal in SF. We are here bitching about the fact that AI shortens a text rather then summarizing it without remarking at the fact that you can actually bitch to the AI directly about this fact and demand the AI to stop shortening the text and start summarizing it.

__loam 2y ago
The problem with this take is that you're conflating different software systems under the same vague AI umbrella. Waymo is a completely different technology than generative AI. Talking about how over a decade and billions of dollars in R&d in self driving is finally bearing fruit has nothing to do with the performance of LLMs. It is true that LLMs appear to be an exciting technology, but it's also delusional to assume they're following a positive trendline. Performance between GPT-3.5 and GPT-4 was like a 25% improvement that took 2500% more resources to train. It's clear that there's diminishing returns to bigger and bigger models, and the trend we've been seeing in the industry is actually smaller models trained for longer periods in an effort to bring down inference costs while maintaining current performance. More intelligent models may require new techniques and technologies that we don't have yet. I'm sure it will get better but the path to improving isn't as obvious as you're making it sound. Making comparisons to Moore's law is also disengenuous because we're actually running into physical limits on how dense we can make chips due to the size of atoms themselves, so past trends for technological development may not continue to bear out.

stareatgoats 2y ago

> Right now the cheering on of AI is overwhelming Perhaps everywhere else. On HN mostly what I see are these fairly shallow dismissals TBH. It is a natural reaction when your own livelihood is affected, old as tech itself to be sure [0]. Still, it's getting tiresome. A new technological revolution is unfolding, and the people best positioned to lead and keep it in check are largely balking. [0] https://en.wikipedia.org/wiki/Luddite https://en.wikipedia.org/wiki/Luddite

parker-3461 2y ago

I would only consider this as “in depth” if and only if the author provided clearer methodology on the exact model used for this write up. It is still unclear to me whether this is a deficiency in all ChatGPT models or this is just one of the many.