8 ms·

The AI bullshit singularity

159 points by hermitcrab 3y ago

horeszko 3y ago
Needle in the haystack. Quality content is the needle, SEO spam is the hay. AI is the hay making machine. Besides curation from a trusted source, what other solutions are there to prevent drowning in hay?
- coffeebeqn 3y ago
  That was already the case. Try to find trustworthy consumer reviews on Google in the last 5 years
nh23423fefe 3y ago
Unoriginal, bad arguments, didn't stop this human from generating this low effort, incoherent text. The bar isn't high. AI will sure eclipse us.
smartmic 3y ago
I broadly agree with that. Repeated training with self-generated data is the technological equivalent of incest and can lead to nothing good.
- rzzzt 3y ago
  Putting washing machine parts in the laundry, irradiating pre-war steel, etc.
- soks86 3y ago
  Only let baby see its own scribbles and scat. Future world leader advisor. Ahem, I hope not.
- chmod775 3y ago
  Humans train on self-generated data all the time, but compared to today's LLMs, humans have superior reasoning ability, which enables them to separate good from bad data. Humans can confirm many things for themselves if need be and our social hierarchies crystallize authority figures which perform curation for others. On top of that humans have a lifetime of experience with the real world, while LLMs rely only on reports of what the real world is like. All in all LLMs being more gullible than your average 6 year old is really not helping. Humans will have to (continue to) curate the data we train LLMs on.
  - jmye 3y ago
    This view of people seems… idealistic, given the number of people who uncritically look to Facebook or Reddit for information (and then happily parrot whatever memes they find) and the number of LLMs that do the same. I think people are substantially worse at figuring out what is “good” data, and I think a huge number of people are/are starting to nominate LLMs as those “authority figures”. And, given that, I think there is a low ceiling for the creators of these tools to clear to make them appealing to users.
    - gorjusborg 3y ago
      > a huge number of people are starting to nominate LLMs as those "authority figures" When LLMs started to go mainstream I said I was not afraid of AI, but was afraid of (unwise) people with AI. This phenomenon is what I foresaw. The worst thinkers are the ones most hungry to outsource their decision making. The fact that LLM output requires good judgement by the recipient to filter the wheat from the chaff, and those with poor judgement skills are most likely to lean on decision tools makes for a potentially volatile situation.
- hermitcrab 3y ago
  Inbreeding certainly didn't go well for the Habsbergs.
  - hermitcrab 3y ago
    Didn't think that would be a controversial topic!
- lewhoo 3y ago
  I think I'll restrain from buying into this until much more testing is conducted. There is information out there suggesting training on synthetic data is at least on par but cheaper.
  - jimbokun 3y ago
    On par at doing what?
    - lewhoo 3y ago
      https://huggingface.co/blog/synthetic-data-save-costs https://huggingface.co/blog/synthetic-data-save-costs At doing whatever the model is suppose to do.
- sebzim4500 3y ago
  Training on self generated data is not necessarily such a problem, see how successful alphazero/muzero is when it is only trained on self play. The key is that you need some kind of external indicator that tells you which generated examples are good and which are bad. In the case of alphazero you get that by simulating games and seeing who wins, in the case of LLMs you will be only taking the generations that are 'successful', e.g. which HN posts get upvoted.
  - vanviegen 3y ago
    Let's indeed hope that it will still be humans doing most of the voting on HN.
- __loam 3y ago
  Are we sampling reality, or are we sampling the photograph?
- twobitshifter 3y ago
  With a mixture of experts approach it’s not incest, we intentionally split the training data to add diversity. Incest is bad because the errors compound on themselves, but this isn’t a problem when the AI parents don’t share the same DNA (problem space.) I see it this way, maybe there’s no new ‘knowledge’ but the AI can apply our collective knowledge better than we can. Within that set of knowledge there are surely discoveries never yet realized based on by the fusion of ideas. Pre-AI we rely on individuals Einstein, Bohr, and Oppenheimer and their associations and studies of each others’ work. With AI, we can fuse the corpus of scientific discoveries into a single entity that we each can communicate with. Maybe the AI lacks the spark of creativity needed to make discovery, but put today’s Einstein in front of it and what would he ask? How much boost would it give him? Einstein said - “I have no special talent. I am only passionately curious.”
29athrowaway 3y ago
When the ELIZA chatbot came out (1964), some users claimed it was intelligent and sentient. Then, they were shown the diagram that describes its behavior. Even after that, they kept insisting it was sentient, etc. Some people just want to believe bullshit.
- hermitcrab 3y ago
  When you interact with an LLM (or Eliza) your brain is doing a lot of heavy lifting, without you even realizing it. I think we tend to infer more intelligence than is there, in the same way we see faces in clouds.
  - pixl97 3y ago
    Watch human behavior long enough and you'll also realize that encoding of social behaviors in our society does a lot of heavy lifting for our brains so we don't have to think. Also, for bonus points define intelligence in a manner that describes simple systems with rudimentary intelligence up to complex systems with higher levels of intelligence than a human. Can you do it, because as of so far no one else has.
    - rocqua 3y ago
      Do you have any links to philosophical attempts at this definition? It feels like it would be a ripe field.
      - pixl97 3y ago
        There is quite a bit of different information, but when any groups cross reference each others work on differences you end up with lines like this "There is no agreed-upon definition of the concept of intelligence neither in psychology nor in philosophy. Experts’ definitions differ widely." Intelligence itself seems to be a bunch of different behaviors and abilities that when combined have emergent behaviors that are difficult to predict and reduce to simple systems.
  - 3y ago
ronald_raygun 3y ago
I always found the idea of infinitely self improving AI to be suspect. Let’s say we have a super smart AI with intelligence 1, and it uses all that to improve itself by 0.5. Then that new 1.5 uses itself to improve by 0.25. Then 0.125, etc etc. obviously it’s always increasing, but it’s not going to have the runaway effect people think.
- ip26 3y ago
  If it improves at a faster rate than humanity, it pulls ahead even if the absolute speed is slow. That's what people are really more worried about, not instant omniscience.
- frotaur 3y ago
  Why should it be (1/2)*i improvement rate ? It could be 1/i ? I don't understand this argument.
  - deleted 3y ago
    [deleted]
- rzzzt 3y ago
  Your AI will end up twice as intelligent - eventually. Were it to improve by 0.5, then 0.333, then 0.25, then 0.2 and so on would be an entirely different matter!
- sebzim4500 3y ago
  Do you have some reason to think that will happen?
  - ronald_raygun 3y ago
    Yeah you could imagine that with a fixed amount of resources that implies a maximum “computational intelligence”. Right now we aren’t close to that limit. But if we get better algorithms, there’s going to be fewer gains as we get towards a finite ceiling.
- 3y ago
jsheard 3y ago
The same goes for image generation models, AI art already has a tendency to veer into the same clichés and those are only going to get reinforced if newer models are trained on newer scrapes which now include the million hyper-derivative AI images being uploaded to places like DeviantArt, Twitter and Pixiv every day. Those vendors who got in early have a moat in the form of untainted scrapes, but they'll eventually need new data to keep up with new subjects/styles/etc.
- rocqua 3y ago
  Are 'scrapes' really the only acceptable source?
  - jsheard 3y ago
    Not always, a few companies like Getty Images are using their internal stock photo libraries to train models rather than slurping up the entire internet. They are uniquely positioned to take that path though, having already been in the stock photo business for decades before AI took off, and for everyone else it's a sisyphean task to manually curate a complete representative dataset that they can be sure isn't contaminated with AI images.
- echelon 3y ago
  This is such a wrong take. Even if the data is 100% synthetic, you can still hill climb to new mountains. If you don't believe me, look at evolution. It doesn't matter if we no longer have 100% human art as input. This is the worst these systems will ever look and feel, and they're only going to improve. I'd be willing to do a longbets on this one.
  - coffeebeqn 3y ago
    How does evolution use synthetic data?
  - rocqua 3y ago
    Hill-climbing requires some measure to optimize. current technology uses 'how well can you predict real text' as this measure. If you change the text you try to predict, it changes the measure you are optimizing. It is far from obvious that this will still improve the actual performance.
gfodor 3y ago
Good cryptographic solutions exist for these problems
- rocqua 3y ago
  Cryptography can make content you create marked. But how would you use it to mark content other (non cooperative) people make?
  - pixl97 3y ago
    Ya anyone that tells you crypto solves the problems of bad actors is delusional or pawning a bill of goods.
    - rocqua 3y ago
      Cryptography is great for amplifying control from controlling access to a single secret to the ability to read information, create 'valid' information, and even to combine new information. What kind tends to limit cryptography from solving problems is that controlling a secret is actually quite difficult.
    - gfodor 3y ago
      No, you can filter bullshit by trusted public keys. This isn’t a new problem. It just now needs to go mainstream.
      - pixl97 3y ago
        "Sorry everyone, my private key got leaked and I had to revoke it, use this public key instead, also buy this crypto" --Really, really Elon Musk Cryptography and human behavior are incompatible.
        gfodor 3y ago
        I mean if you think about it for 5 minutes sure. But look at what bluesky is doing for example around decentralized moderation. It’s not really been an important problem in the global sense to get cryptographic attestations by neural parties for asserting what they have witnessed or analyzed for evidence of falsehoods
opportune 3y ago
The internet is already mostly filled with low quality bullshit though, and GPT-4/Gemini are much better writers than whoever is churning out SEO as we know it now. It's a lazy argument to imply this 1. invalidates the technological achievement 2. prevents iterative improvement a la the singularity. For one, the Internet itself is not bullshit just because a lot of spammers/hustlers put bad content on it to try to make money. And secondly, you can curate datasets... nothings stopping researchers from training LLMs on shitty SEO now, and if they wanted to they could curate datasets going forward to try to prevent LLM-spam from entering the training sets of future models. And finally, people already use reputation/identity/branding and proxies for it as quality filters on the internet. For example, this is an unfamiliar blogger to me and so I entered it with skepticism I wouldn't have with people like Gwern or Lynn Alden. Good writing from people like Gwern and Lynn Alden won't disappear just because LLM content exists on the internet - it just makes reputations and identity (eg to a real human) more important.
- horeszko 3y ago
  Good content won't disappear necessarily, it could however be drowned out by BS as the author states. Its the needle in a haystack problem, where AI is used to increase the size of the haystack, making the needle (i.e. quality content) harder to find.
  - opportune 3y ago
    The Internet is a pull model. I can go to Gwern's website directly and not care that most other websites have crap on them. People choose to use push models for content through meta properties, tiktok, and aggregators like reddit and HN, but nothing is forcing them to. If they push enough bad content, people won't keep using them. Already happened with Facebook and Reddit predecessors, probably happening to Reddit now. It doesn't matter how big the haystack is when you have the ability to go directly to the needle.
    - lmm 3y ago
      Perhaps LLMs will figure out the same thing.
throwuxiytayq 3y ago
> (…) we could quickly end up with an Internet that is mostly bullshit Oh sweet summer child, you must be new here.
rocqua 3y ago
Doesn't that just mean he who can get the best human feedback will get the best ai? If the quality of AI depends on not feeding on 'fake' input, the key becomes getting verified real input. My first thought was that this would create more money in genuine creativity, which would be great. But instead it feels more likely there will be much more telemetry and tracking to determine whether an input is made by you (and thus by a human) or not. "Only trust stuff that can be traced to a specific human" is a great way weed out non-human stuff. But it requires a pretty effective surveillance system...
- pixl97 3y ago
  The Googles, Microsoft, Apples, and Amazon's of the world with access to 'real' humans are the ones with a leg up here. They already have the telemetry, now it's about effectively training models with it.
- monitorlizard 3y ago
  I'm curious about how the economics of human feedback will work out in the long term. I currently work in operations at a data annotation company, and my experience has given me a very pessimistic view of the industry's current state. The MO for these vendors and the AI companies that buy their data seems to be a race to the bottom in price with little concern for quality. The current industry norm is to outsource the work to developing countries, where the cost of living is more in line with what the annotation agencies are willing to pay. While this isn't necessarily problematic for quality in and of itself, it does seem to make it harder to find candidates with the English skills required to generate high-quality RLHF and SFT training data. Furthermore, the pay offered for coding annotators is not competitive with local pay for software engineers, making it challenging to recruit skilled programmers. A lot of coding annotation is done by beginners and students. There is certainly a lot of hype surrounding LLMs and their potential to disrupt various industries — even the US DOD has been scoping out the potential use of LLMs to assist military commanders in strategic decisions. However, if we want these LLMs to consistently perform at an expert level, they need expert-level training data. I worry that producing this data at the quality and scale required may be prohibitively expensive, and could cause a major bottleneck in model improvements long-term.
natch 3y ago
This super low quality, low effort, non-original content does not belong on hacker news imho much less the top of the front page. I’ll take my downvotes.
countWSS 3y ago
It will force more robust filtering and analysis that will allow to determine "quality value" of any information piece and grade its novelty/uniqueness for classification and subsequent training, which will use only high-quality(book-grade) and unique/novel content(excluding copypasted SEO spam).
graemep 3y ago
There is quite a lot of bullshit in human generated content on the internet.
- mseepgood 3y ago
  Exactly. Why do we hold AI to higher standards than ourselves?
  - graemep 3y ago
    For one thing, we use AI to generate answers or outputs we want to rely on. If I use human sources I pick sources I trust. I frequently use multiple sources and check citations. If I still need to do that with "AI", what purpose does it serve?
dkjaudyeqooe 3y ago
The web is well on its way to destroying itself, LLMs are only going to accelerate the process.
- coffeebeqn 3y ago
  Hey maybe we’ll go back to the smaller communities of the 90s, early 00s internet. Any large community will be overflowing with spam.
  - dkjaudyeqooe 3y ago
    It should be clear to people that the general population sucks (eg see most comment sections). We've spent endless time and money sealing ourselves off from the great unwashed only to expose ourselves to them in full force on the internet (plus Russian paid trolls). It doesn't make much sense.
scottbez1 3y ago
Succinctly stated and something that resonates strongly with me. In the last internet revolution (web search), results started high quality because the inputs were high quality - bloggers and others just wanted to document and share knowledge. But over time, many interests (largely commercial) figured out how to game the system with SEO, and quality of search results has decreased as search's incentive structure led to lower quality data being indexed. We're at the start of the LLM revolution now - models are trained on high quality inputs (which may be as rare as "low-background steel" in the future). But the models allow the mass production of lower quality outputs with errors and hallucinations; once those get fed back into new models, are we doomed to decreasing effectiveness of LLMs, just as we've seen with search? Will there be LLMO (LLM optimization) to try to get your commercial interests reflected in the next generation models? I think we've got a few golden years of high quality LLMs before that negative feedback loop really starts to hurt like it did in search.
- celdon25 3y ago
  The comparison is only valid if AI ends up being monopolized. A continuously evolving ecosystem on the other hand has a better chance of adapting to those pressures. I am sure there are search engines that don’t index the SEO crap, but I don’t remember the names, and they have other flaws. Open source AI needs to get a lot of investment for this to be mitigated. Relying on market incentives to drive development without the possibility of forking is too dangerous given the high stakes.
  - piva00 3y ago
    Given enough time, everything seems to go the way of consolidation. I don't have much hope that AI will be different...
- Solvency 3y ago
  This comment/sentiment/idea has already been thoroughly discussed, shared, retweeted, posted, and echoed a couple thousand times online already. Humans like you have been redundantly saying the same things in varying degrees of novelty, like this, for centuries.
aomix 3y ago
I know this comparison has been made https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel That Michigan PhD student dump was by an unauthorized third party and was shut down. But was the pricing reasonable? Are admins of old school message boards sitting on now valuable DB dumps of activity on the board in the form of millions of messages?
ieie3366 3y ago
Have you actually used an LLM? GPT-4 is (well was before turbo) an anti-bullshit machine. You could ask it any question, ”why does my hair look dead if using a hairdryer” and it actually gave an on-point 100% relevant no-bullshit answer. Try googling that, it’s a million seo spam results, none which answer the question
- cookie_monsta 3y ago
  I'm not disagreeing with your point, but this is a really bad example. Google does pretty well with that exact question, including this at #8 (for me at least): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3229938/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3229938/
zamadatix 3y ago
I think bullshit generation is a side effect of any new accelerating technology, not a specific type of new accelerating technology. When we learned to write things down it accelerated our society and our bullshit generation. When we learned to send information electronically it accelerated our society and our bullshit generation. The article already mentions the internet but neglects to mention that is already mostly bullshit prior to LLMs. We still find ways to extract the growing value and move on to the next innovation despite the growing bullshit though.
_giorgio_ 3y ago
This article is bullshit. It's on the first page only because it talks about a very widespread fear, namely that LLMs might actually become dumber because they are trained on their own output. In reality, LLMs are already trained on the output of other LLMs, as specific and well-directed training is much more effective than a disorderly ingestion of text produced by illiterate internet users. Let's not forget the demographic makeup of the typical average user. I think that training on reddit and 4chan messages can be much more harmful than training on text produced by the worst chatbot. Hacker News editors are trying to build a narrative to instill fear and pessimism. "What's the fun in writing on the internet anymore?" explains that everyting can be stolen and rewritten by AIs. And today they talk about LLMs and the whole internet becoming dumber. Tomorrow they'll talk about news falsification and LLMs used to replace writers. https://news.ycombinator.com/item?id=39415878 https://news.ycombinator.com/item?id=39415878
- lewhoo 3y ago
  >In reality, LLMs are already trained on the output of other LLMs That's a fact, but are there any studies on training a LLM on it's own output and not the output of a different LLM ? For instance, chatgpt gets knowledge updates so I understand it must be retrained somehow. What happens if the retraining data contains large patches of its own output ? Has this scenario been explored ?
  - _giorgio_ 3y ago
    Datasets are a dark subject because they contain illegal material. Most gpts, including chatGPT, are trained on the chatGPT outputs. Plus, a lot of chatGPT training material is hand picked by humans.
teraflop 3y ago
I'm almost certain that I've seen this exact phenomenon described in a sci-fi novel written before LLMs existed. I thought it was Neal Stephenson's Anathem, but a quick skim through the text didn't turn anything up. Anybody else know what I might be thinking of?
- empathy_m 3y ago
  Yes it's a major plot point in Anathem. The Internet was so badly polluted with machine generated garbage that a special caste of techno priests, the Ita, evolved with the mission of doing machine augmented web searches to find valid knowledge among the vast noise.
pton_xd 3y ago
LLMs are increasingly trained on synthetic data. A LLM can learn logic, and ultimately the rules governing physics and everything else, without human input. The first versions being trained on "human knowledge" seems kind of like a proof-of-concept. Future iterations will be much, much smarter.
- deleted 3y ago
  [deleted]
mrkramer 3y ago
As AI advances, there will be AI and algorithms that will check LLMs and their output, sort of LLMs' verifier. It is foolish to think that LLMs won't improve to the point where there are almost no or very low hallucinations and false information. We are just at the beginning of LLMs and generative AI.
- hermitcrab 3y ago
  How do you automatically verify that an LLM isn't hallucinating?
  - mrkramer 3y ago
    Idk, ask LLMs researchers....they will figure it out sooner or later.
    - hermitcrab 3y ago
      Your faith is touching.
    - Jensson 3y ago
      When people don't know how to do something but think it is easy, then it can often take 50 years or more for that to happen. Has happened before in this field.
- vrighter 3y ago
  "The queen is ________" The model predicts a 99% chance that the next word is "alive" and a 1% chance that it's "dead". The llm calculates probabilities. How does it actually choose the word? It throws a weighted die. It literally chooses one at random (albeit on a custom probability distribution). So tell me, what how can you eliminate hallucinations from something that is literally designed to pick stuff at random? Hallucinations will never be removed from these types of llms. Hallucinations are fundamental to how they work. In the sense that even the "good" outputs are hallucinations picked out at random from a probability distribution. Any company that says they can control hallucinations, in any way, is flat out lying.
gdiamos 3y ago
Technologies usually follow a learning curve eg wrights law, moores law, LLM scaling laws AI seems to add another dimension of explosive hype
gumby 3y ago
I lived (and was gainfully employed working on AI) during the mid 80s AI winter. This time feels the same. Recently a potential investor asked if we were using AI in our product. I said, “no, not at all, just a neural network.” They seemed satisfied.
MichaelMug 3y ago
You know I've read a lot here on HN about AI and our future. I'd like to hear more from the folks actually in the AI industry the researchers and developers implementing LLMs, diffusion models, etc. Will high quality training data be needed for incremental improvements? What is the upper bound of improvements, and what is it contingent on? Compute, training data, etc.
GreedClarifies 3y ago
Jesus Christ. If all I did was read hackernews comments I would think that these magical algorithms which have the potential for massive positive change don't exist and that the world is coming to and end. I get it, engineering trains us to look for failure modes, but my god try to have a little amazement at the progress. FFS.
valine 3y ago
I can read my own writings without overfitting the neurons in my brain. The key I think is contextualization, something LLMs are great at already. The open question is how to utilize that contextualization ability during training. The argument that LLMs can’t possibly scale because of data contamination falls apart the moment we discover a method to incorporate context-learning into the training loop.
davesque 3y ago
On the bright side, it might force those engaged in research to figure out just what exactly it is that makes their models work to begin with. Turns out knowledge is about more than just modeling statistics.
danielovichdk 3y ago
I have absolutely no idea why people are so worried. There is a human choice in a lot of matters, especially when it comes to how and why one perceives quality. Quality in literature. Food. Arts. Fashion. Blogs. Programming. Acting. And all such things. What I believe, based on a both cultural and philosophy stance at where the world is and where its going, is that, at best, so-called AI will push an even larger human segment towards finding the truth for oneself in a different and orginal aspect in terms of being human and how one perceives reality. I think that the intelligent and thoughtful individual will find plenty of ways to nuture one's own abilities towards quality, which has always been the only true real value you can measure yourself against. No one will stand a chance against the fluff and the obvious stupidity of "my ai will call your ai" and therfore, the only real direction to follow is the one that gives you an edge for yourself based on your fundamental skills you learn through years and years of experience and personality. The reason great books are great is not that it's written. It's that it sips through the words and sentences, by their very human authors, that you can feel them inside of you. There will be, is my belief, a larger calling in the world for removing oneself from the fast paced results which has absolutely no nerve or soul. I am not afraid at all. I only feel sorry for the humans that has nothing to give in themselves to obtain a larger meaning with their time spent, in whatever profession or life they pursue.
invig 3y ago
The web already contains vastly more information than you could ever even begin to read or look at in a lifetime. Most of it is total garbage. You have to use a trust filter to find things worth looking at. You're arguing that there isn't a trust filter that can do that, but there will be, and it will probably be an AI.
jwie 3y ago
The Dead Internet Theory was only slightly ahead of its time. Used to be real people pretended to be girls on the internet. These days I can’t even get an honest real fake person pretending to be an attractive female on LinkedIn.
zbowling 3y ago
This sounds like bullshit generated by an LLM. How will we ever know for sure?
- jimbokun 3y ago
  Well, if so, it only strengthens the argument.
aj7 3y ago
Well, for now, anyway. I went to college in the 70’s. What strikes me is, once obviously wrong hallucinations are eliminated, LLM’s will do exactly what untalented liberal arts students did with their time, but with some useful stuff tacked on.
rambambram 3y ago
Of course it's not going to improve on itself. That's what you all are here for - happily interacting with LLM's. Doing the finetuning and improving it with yet another dose of humanity, albeit with more direct interaction this time. And everybody's asking what AI is going to be and can do for them, all the while freely working for it. Don't ask what AI can do for you, just do for AI what is asked of you.
pistachiopro 3y ago
LLMs are being trained on a smaller and smaller percentage of human prose. Right now it seems like code is the best source for the bulk of an LLM's diet, but it's also looking likely that synthetic math text will be even better. The structured reasoning of code and math seems to be what actually makes these big LLMs "smart." Once you've trained a smart LLM, it seems to take a relatively small amount of hand-curated human prose to fine tune it into talking like a human. Unfortunately this article feels like the wishful thinking of someone who is afraid of the changes LLMs are bringing and hasn't done much research.
- EasyMark 3y ago
  It seems to me like archive.org and the major book publishers are sitting on a gold mine(at least up to 2022), but I haven't seen anyone saying the same, so maybe I just don't know enough about LLM.
hermitcrab 3y ago
The article appears to have been delisted by HN. Not sure why. 'Big AI' didn't like it, perhaps?
- pton_xd 3y ago
  When an article crosses some threshhold of replies to upvotes per unit of time it gets deranked, presumably to discourage political / flame-war topics. Eg, this article still shows up on the second page. After a while, when the replies slow down, it will move back to the first page (if it's still getting upvotes).
  - hermitcrab 3y ago
    Didn't know that. Thanks.
maaaaattttt 3y ago
The fact that internet is going to be filled with AI content is likely to happen. But for the rest I think it’s the classical bias science fiction has when predicting the future. More of the same. Or the linear transposition of the now (for a lack of better wording). Think CRT screens alongside flying cars. One could argue that LLMs have some form of intelligence already even though I believe it falls more under “very good intuition”. Even if the internet is filled with trash, we still have (and will continue to do so I hope) a lot of content of high quality available (all the books, podcasts, videos, etc. produced up until now). If you take human intelligence, it doesn’t require the whole internet to start to get smart. A lot of interactions with the real world and some good books/videos should be enough to get to AGI (and start the dreaded feedback loop). We just haven’t found a way to achieve it yet (AFAIK).
Eliah_Lakhin 3y ago
I think that people's believe in LLM is intelligence is closely tied to mistaken association between intelligence and the ability to speak. Parrots can speak, but they cannot reason. Moreover, evolutionary, birds learned to mimic the speech to fool other species. To fool in such way that other species would think that parrots are of the same specie. We, the humans, are smart enough to recognise that even though parrots can talk, they are not as smart as we are. Unfortunately for us, LLMs are much more advanced things than parrots, and they are capable to fool broad masses pretending that LLMs are of the human specie with all inherent features including the reasoning intelligence (that we cannot easily test externally). This is unfortunate, because such false believes slow down actual scientific progress towards the natural intelligence researches, and towards creating of the true reasoning artificial intelligence. Btw, I wouldn't be surprised that if one day the AGI created, it will not be able to speak at all, nor recognise the images.
- krono 3y ago
  I believe you are vastly underestimating the capabilities of parrots or similarly intelligent birds. At least when my pet parrot manages to fool me into whatever it is he wants to get out of me that time, he does it for his own primal benefit and not because it was programmed to adhere to some strict set of ethic and moral boundaries set by legal requirements and someone else's idea of how people should think and behave.
  - smackeyacky 3y ago
    The smart behaviour of parrots and their speech synthesis are two different phenomenon. Parrots are definitely smart, and parrots can definitely speak, but they can't learn to speak beyond what they are trained to do. It's not like a parrot has ever spontaneously put together a coherent sentence independently. LLMs aren't even that smart. Not even close.
    - krono 3y ago
      Absolutely, I also replied to the OP's response below that I failed to specify that I was arguing parrots' ability to reason which I do believe exists to a great extend. Regardless, speech is just another means of communication. In the end it doesn't really matter if I use perfectly articulated Swahili or just scream "aaar" at you for a few seconds as long as I get the cheeseburger with extra cheese I want from you. These parrots, much like children, just push your buttons and are quite good at finding (or negotiating) the right pattern of things to do and sounds to make to get to a certain outcome.
lifeisstillgood 3y ago
The biggest threat here is not waves of bullshit, that’s nothing new. It’s knowing who to trust. One of the defining factors under the Stasi in East Germany, it was not that “ordinary people” could not recognise what they heard on the radio was bullshit, but that they could not know who to trust to say “that’s bullshit”. Every family has a Stasi informer, so whilst everyone knew the regieme was lying there was no critical mass event. There are a dozen technical solutions to “bullshit”. We just need to ensure we have institutions that support us standing up to it.
bitshiftfaced 3y ago
The assumption here is that future iterations of the technology require "BS"-riddled datasets. I question that assumption, both because the technology probably can improve solely with existing datasets and because we don't know that "synthetic" data isn't able to improve things.
sieste 3y ago
Isn't this really good news for artists? When everything you see is generic shit, it will be so much easier for truly creative thoughts, expressed through high quality writing, based on genuine experiences in the real world, to stand out. Also, this will put pressure on human creativity to produce weirder stuff that can't be produced by simply paraphrasing and remixing known ideas. Mass photography was bad news for the portrait painting industry, but forced painters to think outside known concepts and explore crazy abstract art. I totally look forward to whatever humans come up with next, that can't be easily generated by a computer program.
bulbosaur123 3y ago
SEO is dead. It will all be pay to play from now on. Want exposure? Open your wallet. Paid marketing will be the final bottleneck as problem of content has been resolved forever.
noduerme 3y ago
Ugh, the guy who thinks "prompt editor" is the job title of the future, and that people in every field will take classes on prompt editing. Gag. You know, there's no replacement for knowing how to do something yourself. And no, you cannot be as good creatively if you haven't learned a craft. (And no, prompt editing is not a craft any more than ordering food in a restaurant is cooking. Could you even describe in a useful way what was wrong with a soup you were served if you had never learned to cook?)
imacomputer 3y ago
This is a weird thread. I feel like there are many people who have no clue what they are talking about other than a few buzzwords, and make huge generalizations about AI or intelligence in general to _will their opinion into reality_. These statements are biased by either fear of loosing their job or fear of loosing on some investments on AI. Obviously, the enshittification threat as the author points out is significant. It will be interesting problem to deal with in the future, but until then I feel like no one knows when or what that will look like. At this rate, no one can predict what AI will bring us 6 months in the future even!
hermitcrab 3y ago
The argument that the problems of AI are going to be solved with MOAR AI, sounds suspiciously like the argument that the solution to America's gun problem will be solved with MOAR GUNS.
symbolicAGI 3y ago
Consider LLM code generation. It would an ideal domain for simulated data. For example, a version of popular software is released with new functions. A framework could simulate numerous plausible code snippets that exercise the new release, including edge cases. The code snippets would be amply commented, and proven to work in a test harness. This method is the opposite of depending upon the internet for training data. And this method is being used to train AI robots, that have no available internet training data.
evolextra 3y ago
[dead]
Luisvel 3y ago
Not sure why I expected something much more insighful when opening the link. The knowledge base that LLMs have today, is probably enough to derive very high quality content/knowledge. Data and information coming "from outside" won't cease to flow, but will otherwise increase. On the other hand, I see value in somehow indexing/marking human generated content, or at least human supervised/approved content.

159 points by hermitcrab 3y ago

Needle in the haystack. Quality content is the needle, SEO spam is the hay. AI is the hay making machine. Besides curation from a trusted source, what other solutions are there to prevent drowning in hay?

coffeebeqn 3y ago
That was already the case. Try to find trustworthy consumer reviews on Google in the last 5 years

nh23423fefe 3y ago

Unoriginal, bad arguments, didn't stop this human from generating this low effort, incoherent text. The bar isn't high. AI will sure eclipse us.

smartmic 3y ago

I broadly agree with that. Repeated training with self-generated data is the technological equivalent of incest and can lead to nothing good.

rzzzt 3y ago
Putting washing machine parts in the laundry, irradiating pre-war steel, etc.
soks86 3y ago
Only let baby see its own scribbles and scat. Future world leader advisor. Ahem, I hope not.
chmod775 3y ago
Humans train on self-generated data all the time, but compared to today's LLMs, humans have superior reasoning ability, which enables them to separate good from bad data. Humans can confirm many things for themselves if need be and our social hierarchies crystallize authority figures which perform curation for others. On top of that humans have a lifetime of experience with the real world, while LLMs rely only on reports of what the real world is like. All in all LLMs being more gullible than your average 6 year old is really not helping. Humans will have to (continue to) curate the data we train LLMs on.
- jmye 3y ago
  This view of people seems… idealistic, given the number of people who uncritically look to Facebook or Reddit for information (and then happily parrot whatever memes they find) and the number of LLMs that do the same. I think people are substantially worse at figuring out what is “good” data, and I think a huge number of people are/are starting to nominate LLMs as those “authority figures”. And, given that, I think there is a low ceiling for the creators of these tools to clear to make them appealing to users.
  - gorjusborg 3y ago
    > a huge number of people are starting to nominate LLMs as those "authority figures" When LLMs started to go mainstream I said I was not afraid of AI, but was afraid of (unwise) people with AI. This phenomenon is what I foresaw. The worst thinkers are the ones most hungry to outsource their decision making. The fact that LLM output requires good judgement by the recipient to filter the wheat from the chaff, and those with poor judgement skills are most likely to lean on decision tools makes for a potentially volatile situation.
hermitcrab 3y ago
Inbreeding certainly didn't go well for the Habsbergs.
- hermitcrab 3y ago
  Didn't think that would be a controversial topic!
lewhoo 3y ago
I think I'll restrain from buying into this until much more testing is conducted. There is information out there suggesting training on synthetic data is at least on par but cheaper.
- jimbokun 3y ago
  On par at doing what?
  - lewhoo 3y ago
    https://huggingface.co/blog/synthetic-data-save-costs https://huggingface.co/blog/synthetic-data-save-costs At doing whatever the model is suppose to do.
sebzim4500 3y ago
Training on self generated data is not necessarily such a problem, see how successful alphazero/muzero is when it is only trained on self play. The key is that you need some kind of external indicator that tells you which generated examples are good and which are bad. In the case of alphazero you get that by simulating games and seeing who wins, in the case of LLMs you will be only taking the generations that are 'successful', e.g. which HN posts get upvoted.
- vanviegen 3y ago
  Let's indeed hope that it will still be humans doing most of the voting on HN.
__loam 3y ago
Are we sampling reality, or are we sampling the photograph?
twobitshifter 3y ago
With a mixture of experts approach it’s not incest, we intentionally split the training data to add diversity. Incest is bad because the errors compound on themselves, but this isn’t a problem when the AI parents don’t share the same DNA (problem space.) I see it this way, maybe there’s no new ‘knowledge’ but the AI can apply our collective knowledge better than we can. Within that set of knowledge there are surely discoveries never yet realized based on by the fusion of ideas. Pre-AI we rely on individuals Einstein, Bohr, and Oppenheimer and their associations and studies of each others’ work. With AI, we can fuse the corpus of scientific discoveries into a single entity that we each can communicate with. Maybe the AI lacks the spark of creativity needed to make discovery, but put today’s Einstein in front of it and what would he ask? How much boost would it give him? Einstein said - “I have no special talent. I am only passionately curious.”

29athrowaway 3y ago

When the ELIZA chatbot came out (1964), some users claimed it was intelligent and sentient. Then, they were shown the diagram that describes its behavior. Even after that, they kept insisting it was sentient, etc. Some people just want to believe bullshit.

hermitcrab 3y ago
When you interact with an LLM (or Eliza) your brain is doing a lot of heavy lifting, without you even realizing it. I think we tend to infer more intelligence than is there, in the same way we see faces in clouds.
- pixl97 3y ago
  Watch human behavior long enough and you'll also realize that encoding of social behaviors in our society does a lot of heavy lifting for our brains so we don't have to think. Also, for bonus points define intelligence in a manner that describes simple systems with rudimentary intelligence up to complex systems with higher levels of intelligence than a human. Can you do it, because as of so far no one else has.
  - rocqua 3y ago
    Do you have any links to philosophical attempts at this definition? It feels like it would be a ripe field.
    - pixl97 3y ago
      There is quite a bit of different information, but when any groups cross reference each others work on differences you end up with lines like this "There is no agreed-upon definition of the concept of intelligence neither in psychology nor in philosophy. Experts’ definitions differ widely." Intelligence itself seems to be a bunch of different behaviors and abilities that when combined have emergent behaviors that are difficult to predict and reduce to simple systems.
- 3y ago

ronald_raygun 3y ago

I always found the idea of infinitely self improving AI to be suspect. Let’s say we have a super smart AI with intelligence 1, and it uses all that to improve itself by 0.5. Then that new 1.5 uses itself to improve by 0.25. Then 0.125, etc etc. obviously it’s always increasing, but it’s not going to have the runaway effect people think.

ip26 3y ago
If it improves at a faster rate than humanity, it pulls ahead even if the absolute speed is slow. That's what people are really more worried about, not instant omniscience.
frotaur 3y ago
Why should it be (1/2)*i improvement rate ? It could be 1/i ? I don't understand this argument.
- deleted 3y ago
  [deleted]
rzzzt 3y ago
Your AI will end up twice as intelligent - eventually. Were it to improve by 0.5, then 0.333, then 0.25, then 0.2 and so on would be an entirely different matter!
sebzim4500 3y ago
Do you have some reason to think that will happen?
- ronald_raygun 3y ago
  Yeah you could imagine that with a fixed amount of resources that implies a maximum “computational intelligence”. Right now we aren’t close to that limit. But if we get better algorithms, there’s going to be fewer gains as we get towards a finite ceiling.
3y ago

jsheard 3y ago

The same goes for image generation models, AI art already has a tendency to veer into the same clichés and those are only going to get reinforced if newer models are trained on newer scrapes which now include the million hyper-derivative AI images being uploaded to places like DeviantArt, Twitter and Pixiv every day. Those vendors who got in early have a moat in the form of untainted scrapes, but they'll eventually need new data to keep up with new subjects/styles/etc.

rocqua 3y ago
Are 'scrapes' really the only acceptable source?
- jsheard 3y ago
  Not always, a few companies like Getty Images are using their internal stock photo libraries to train models rather than slurping up the entire internet. They are uniquely positioned to take that path though, having already been in the stock photo business for decades before AI took off, and for everyone else it's a sisyphean task to manually curate a complete representative dataset that they can be sure isn't contaminated with AI images.
echelon 3y ago
This is such a wrong take. Even if the data is 100% synthetic, you can still hill climb to new mountains. If you don't believe me, look at evolution. It doesn't matter if we no longer have 100% human art as input. This is the worst these systems will ever look and feel, and they're only going to improve. I'd be willing to do a longbets on this one.
- coffeebeqn 3y ago
  How does evolution use synthetic data?
- rocqua 3y ago
  Hill-climbing requires some measure to optimize. current technology uses 'how well can you predict real text' as this measure. If you change the text you try to predict, it changes the measure you are optimizing. It is far from obvious that this will still improve the actual performance.

gfodor 3y ago

Good cryptographic solutions exist for these problems

rocqua 3y ago
Cryptography can make content you create marked. But how would you use it to mark content other (non cooperative) people make?
- pixl97 3y ago
  Ya anyone that tells you crypto solves the problems of bad actors is delusional or pawning a bill of goods.
  - rocqua 3y ago
    Cryptography is great for amplifying control from controlling access to a single secret to the ability to read information, create 'valid' information, and even to combine new information. What kind tends to limit cryptography from solving problems is that controlling a secret is actually quite difficult.
  - gfodor 3y ago
    No, you can filter bullshit by trusted public keys. This isn’t a new problem. It just now needs to go mainstream.
    - pixl97 3y ago
      "Sorry everyone, my private key got leaked and I had to revoke it, use this public key instead, also buy this crypto" --Really, really Elon Musk Cryptography and human behavior are incompatible.
      - gfodor 3y ago
        I mean if you think about it for 5 minutes sure. But look at what bluesky is doing for example around decentralized moderation. It’s not really been an important problem in the global sense to get cryptographic attestations by neural parties for asserting what they have witnessed or analyzed for evidence of falsehoods

opportune 3y ago

The internet is already mostly filled with low quality bullshit though, and GPT-4/Gemini are much better writers than whoever is churning out SEO as we know it now. It's a lazy argument to imply this 1. invalidates the technological achievement 2. prevents iterative improvement a la the singularity. For one, the Internet itself is not bullshit just because a lot of spammers/hustlers put bad content on it to try to make money. And secondly, you can curate datasets... nothings stopping researchers from training LLMs on shitty SEO now, and if they wanted to they could curate datasets going forward to try to prevent LLM-spam from entering the training sets of future models. And finally, people already use reputation/identity/branding and proxies for it as quality filters on the internet. For example, this is an unfamiliar blogger to me and so I entered it with skepticism I wouldn't have with people like Gwern or Lynn Alden. Good writing from people like Gwern and Lynn Alden won't disappear just because LLM content exists on the internet - it just makes reputations and identity (eg to a real human) more important.

horeszko 3y ago
Good content won't disappear necessarily, it could however be drowned out by BS as the author states. Its the needle in a haystack problem, where AI is used to increase the size of the haystack, making the needle (i.e. quality content) harder to find.
- opportune 3y ago
  The Internet is a pull model. I can go to Gwern's website directly and not care that most other websites have crap on them. People choose to use push models for content through meta properties, tiktok, and aggregators like reddit and HN, but nothing is forcing them to. If they push enough bad content, people won't keep using them. Already happened with Facebook and Reddit predecessors, probably happening to Reddit now. It doesn't matter how big the haystack is when you have the ability to go directly to the needle.
  - lmm 3y ago
    Perhaps LLMs will figure out the same thing.

throwuxiytayq 3y ago

> (…) we could quickly end up with an Internet that is mostly bullshit Oh sweet summer child, you must be new here.

rocqua 3y ago

Doesn't that just mean he who can get the best human feedback will get the best ai? If the quality of AI depends on not feeding on 'fake' input, the key becomes getting verified real input. My first thought was that this would create more money in genuine creativity, which would be great. But instead it feels more likely there will be much more telemetry and tracking to determine whether an input is made by you (and thus by a human) or not. "Only trust stuff that can be traced to a specific human" is a great way weed out non-human stuff. But it requires a pretty effective surveillance system...

pixl97 3y ago
The Googles, Microsoft, Apples, and Amazon's of the world with access to 'real' humans are the ones with a leg up here. They already have the telemetry, now it's about effectively training models with it.
monitorlizard 3y ago
I'm curious about how the economics of human feedback will work out in the long term. I currently work in operations at a data annotation company, and my experience has given me a very pessimistic view of the industry's current state. The MO for these vendors and the AI companies that buy their data seems to be a race to the bottom in price with little concern for quality. The current industry norm is to outsource the work to developing countries, where the cost of living is more in line with what the annotation agencies are willing to pay. While this isn't necessarily problematic for quality in and of itself, it does seem to make it harder to find candidates with the English skills required to generate high-quality RLHF and SFT training data. Furthermore, the pay offered for coding annotators is not competitive with local pay for software engineers, making it challenging to recruit skilled programmers. A lot of coding annotation is done by beginners and students. There is certainly a lot of hype surrounding LLMs and their potential to disrupt various industries — even the US DOD has been scoping out the potential use of LLMs to assist military commanders in strategic decisions. However, if we want these LLMs to consistently perform at an expert level, they need expert-level training data. I worry that producing this data at the quality and scale required may be prohibitively expensive, and could cause a major bottleneck in model improvements long-term.

natch 3y ago

This super low quality, low effort, non-original content does not belong on hacker news imho much less the top of the front page. I’ll take my downvotes.

countWSS 3y ago

It will force more robust filtering and analysis that will allow to determine "quality value" of any information piece and grade its novelty/uniqueness for classification and subsequent training, which will use only high-quality(book-grade) and unique/novel content(excluding copypasted SEO spam).

graemep 3y ago

There is quite a lot of bullshit in human generated content on the internet.

mseepgood 3y ago
Exactly. Why do we hold AI to higher standards than ourselves?
- graemep 3y ago
  For one thing, we use AI to generate answers or outputs we want to rely on. If I use human sources I pick sources I trust. I frequently use multiple sources and check citations. If I still need to do that with "AI", what purpose does it serve?

dkjaudyeqooe 3y ago

The web is well on its way to destroying itself, LLMs are only going to accelerate the process.

coffeebeqn 3y ago
Hey maybe we’ll go back to the smaller communities of the 90s, early 00s internet. Any large community will be overflowing with spam.
- dkjaudyeqooe 3y ago
  It should be clear to people that the general population sucks (eg see most comment sections). We've spent endless time and money sealing ourselves off from the great unwashed only to expose ourselves to them in full force on the internet (plus Russian paid trolls). It doesn't make much sense.

scottbez1 3y ago

Succinctly stated and something that resonates strongly with me. In the last internet revolution (web search), results started high quality because the inputs were high quality - bloggers and others just wanted to document and share knowledge. But over time, many interests (largely commercial) figured out how to game the system with SEO, and quality of search results has decreased as search's incentive structure led to lower quality data being indexed. We're at the start of the LLM revolution now - models are trained on high quality inputs (which may be as rare as "low-background steel" in the future). But the models allow the mass production of lower quality outputs with errors and hallucinations; once those get fed back into new models, are we doomed to decreasing effectiveness of LLMs, just as we've seen with search? Will there be LLMO (LLM optimization) to try to get your commercial interests reflected in the next generation models? I think we've got a few golden years of high quality LLMs before that negative feedback loop really starts to hurt like it did in search.

celdon25 3y ago
The comparison is only valid if AI ends up being monopolized. A continuously evolving ecosystem on the other hand has a better chance of adapting to those pressures. I am sure there are search engines that don’t index the SEO crap, but I don’t remember the names, and they have other flaws. Open source AI needs to get a lot of investment for this to be mitigated. Relying on market incentives to drive development without the possibility of forking is too dangerous given the high stakes.
- piva00 3y ago
  Given enough time, everything seems to go the way of consolidation. I don't have much hope that AI will be different...
Solvency 3y ago
This comment/sentiment/idea has already been thoroughly discussed, shared, retweeted, posted, and echoed a couple thousand times online already. Humans like you have been redundantly saying the same things in varying degrees of novelty, like this, for centuries.

aomix 3y ago

I know this comparison has been made https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel That Michigan PhD student dump was by an unauthorized third party and was shut down. But was the pricing reasonable? Are admins of old school message boards sitting on now valuable DB dumps of activity on the board in the form of millions of messages?

ieie3366 3y ago

Have you actually used an LLM? GPT-4 is (well was before turbo) an anti-bullshit machine. You could ask it any question, ”why does my hair look dead if using a hairdryer” and it actually gave an on-point 100% relevant no-bullshit answer. Try googling that, it’s a million seo spam results, none which answer the question

cookie_monsta 3y ago
I'm not disagreeing with your point, but this is a really bad example. Google does pretty well with that exact question, including this at #8 (for me at least): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3229938/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3229938/

zamadatix 3y ago

I think bullshit generation is a side effect of any new accelerating technology, not a specific type of new accelerating technology. When we learned to write things down it accelerated our society and our bullshit generation. When we learned to send information electronically it accelerated our society and our bullshit generation. The article already mentions the internet but neglects to mention that is already mostly bullshit prior to LLMs. We still find ways to extract the growing value and move on to the next innovation despite the growing bullshit though.

_giorgio_ 3y ago

This article is bullshit. It's on the first page only because it talks about a very widespread fear, namely that LLMs might actually become dumber because they are trained on their own output. In reality, LLMs are already trained on the output of other LLMs, as specific and well-directed training is much more effective than a disorderly ingestion of text produced by illiterate internet users. Let's not forget the demographic makeup of the typical average user. I think that training on reddit and 4chan messages can be much more harmful than training on text produced by the worst chatbot. Hacker News editors are trying to build a narrative to instill fear and pessimism. "What's the fun in writing on the internet anymore?" explains that everyting can be stolen and rewritten by AIs. And today they talk about LLMs and the whole internet becoming dumber. Tomorrow they'll talk about news falsification and LLMs used to replace writers. https://news.ycombinator.com/item?id=39415878 https://news.ycombinator.com/item?id=39415878

lewhoo 3y ago
>In reality, LLMs are already trained on the output of other LLMs That's a fact, but are there any studies on training a LLM on it's own output and not the output of a different LLM ? For instance, chatgpt gets knowledge updates so I understand it must be retrained somehow. What happens if the retraining data contains large patches of its own output ? Has this scenario been explored ?
- _giorgio_ 3y ago
  Datasets are a dark subject because they contain illegal material. Most gpts, including chatGPT, are trained on the chatGPT outputs. Plus, a lot of chatGPT training material is hand picked by humans.

teraflop 3y ago

I'm almost certain that I've seen this exact phenomenon described in a sci-fi novel written before LLMs existed. I thought it was Neal Stephenson's Anathem, but a quick skim through the text didn't turn anything up. Anybody else know what I might be thinking of?

empathy_m 3y ago
Yes it's a major plot point in Anathem. The Internet was so badly polluted with machine generated garbage that a special caste of techno priests, the Ita, evolved with the mission of doing machine augmented web searches to find valid knowledge among the vast noise.

pton_xd 3y ago

LLMs are increasingly trained on synthetic data. A LLM can learn logic, and ultimately the rules governing physics and everything else, without human input. The first versions being trained on "human knowledge" seems kind of like a proof-of-concept. Future iterations will be much, much smarter.

deleted 3y ago
[deleted]

mrkramer 3y ago

As AI advances, there will be AI and algorithms that will check LLMs and their output, sort of LLMs' verifier. It is foolish to think that LLMs won't improve to the point where there are almost no or very low hallucinations and false information. We are just at the beginning of LLMs and generative AI.

hermitcrab 3y ago
How do you automatically verify that an LLM isn't hallucinating?
- mrkramer 3y ago
  Idk, ask LLMs researchers....they will figure it out sooner or later.
  - hermitcrab 3y ago
    Your faith is touching.
  - Jensson 3y ago
    When people don't know how to do something but think it is easy, then it can often take 50 years or more for that to happen. Has happened before in this field.
vrighter 3y ago
"The queen is ________" The model predicts a 99% chance that the next word is "alive" and a 1% chance that it's "dead". The llm calculates probabilities. How does it actually choose the word? It throws a weighted die. It literally chooses one at random (albeit on a custom probability distribution). So tell me, what how can you eliminate hallucinations from something that is literally designed to pick stuff at random? Hallucinations will never be removed from these types of llms. Hallucinations are fundamental to how they work. In the sense that even the "good" outputs are hallucinations picked out at random from a probability distribution. Any company that says they can control hallucinations, in any way, is flat out lying.

gdiamos 3y ago

Technologies usually follow a learning curve eg wrights law, moores law, LLM scaling laws AI seems to add another dimension of explosive hype

gumby 3y ago

I lived (and was gainfully employed working on AI) during the mid 80s AI winter. This time feels the same. Recently a potential investor asked if we were using AI in our product. I said, “no, not at all, just a neural network.” They seemed satisfied.

MichaelMug 3y ago

You know I've read a lot here on HN about AI and our future. I'd like to hear more from the folks actually in the AI industry the researchers and developers implementing LLMs, diffusion models, etc. Will high quality training data be needed for incremental improvements? What is the upper bound of improvements, and what is it contingent on? Compute, training data, etc.

GreedClarifies 3y ago

Jesus Christ. If all I did was read hackernews comments I would think that these magical algorithms which have the potential for massive positive change don't exist and that the world is coming to and end. I get it, engineering trains us to look for failure modes, but my god try to have a little amazement at the progress. FFS.

valine 3y ago

I can read my own writings without overfitting the neurons in my brain. The key I think is contextualization, something LLMs are great at already. The open question is how to utilize that contextualization ability during training. The argument that LLMs can’t possibly scale because of data contamination falls apart the moment we discover a method to incorporate context-learning into the training loop.

davesque 3y ago

On the bright side, it might force those engaged in research to figure out just what exactly it is that makes their models work to begin with. Turns out knowledge is about more than just modeling statistics.

danielovichdk 3y ago

I have absolutely no idea why people are so worried. There is a human choice in a lot of matters, especially when it comes to how and why one perceives quality. Quality in literature. Food. Arts. Fashion. Blogs. Programming. Acting. And all such things. What I believe, based on a both cultural and philosophy stance at where the world is and where its going, is that, at best, so-called AI will push an even larger human segment towards finding the truth for oneself in a different and orginal aspect in terms of being human and how one perceives reality. I think that the intelligent and thoughtful individual will find plenty of ways to nuture one's own abilities towards quality, which has always been the only true real value you can measure yourself against. No one will stand a chance against the fluff and the obvious stupidity of "my ai will call your ai" and therfore, the only real direction to follow is the one that gives you an edge for yourself based on your fundamental skills you learn through years and years of experience and personality. The reason great books are great is not that it's written. It's that it sips through the words and sentences, by their very human authors, that you can feel them inside of you. There will be, is my belief, a larger calling in the world for removing oneself from the fast paced results which has absolutely no nerve or soul. I am not afraid at all. I only feel sorry for the humans that has nothing to give in themselves to obtain a larger meaning with their time spent, in whatever profession or life they pursue.

invig 3y ago

The web already contains vastly more information than you could ever even begin to read or look at in a lifetime. Most of it is total garbage. You have to use a trust filter to find things worth looking at. You're arguing that there isn't a trust filter that can do that, but there will be, and it will probably be an AI.

jwie 3y ago

The Dead Internet Theory was only slightly ahead of its time. Used to be real people pretended to be girls on the internet. These days I can’t even get an honest real fake person pretending to be an attractive female on LinkedIn.

zbowling 3y ago

This sounds like bullshit generated by an LLM. How will we ever know for sure?

jimbokun 3y ago
Well, if so, it only strengthens the argument.

aj7 3y ago

Well, for now, anyway. I went to college in the 70’s. What strikes me is, once obviously wrong hallucinations are eliminated, LLM’s will do exactly what untalented liberal arts students did with their time, but with some useful stuff tacked on.

rambambram 3y ago

Of course it's not going to improve on itself. That's what you all are here for - happily interacting with LLM's. Doing the finetuning and improving it with yet another dose of humanity, albeit with more direct interaction this time. And everybody's asking what AI is going to be and can do for them, all the while freely working for it. Don't ask what AI can do for you, just do for AI what is asked of you.

pistachiopro 3y ago

LLMs are being trained on a smaller and smaller percentage of human prose. Right now it seems like code is the best source for the bulk of an LLM's diet, but it's also looking likely that synthetic math text will be even better. The structured reasoning of code and math seems to be what actually makes these big LLMs "smart." Once you've trained a smart LLM, it seems to take a relatively small amount of hand-curated human prose to fine tune it into talking like a human. Unfortunately this article feels like the wishful thinking of someone who is afraid of the changes LLMs are bringing and hasn't done much research.

EasyMark 3y ago
It seems to me like archive.org and the major book publishers are sitting on a gold mine(at least up to 2022), but I haven't seen anyone saying the same, so maybe I just don't know enough about LLM.

hermitcrab 3y ago

The article appears to have been delisted by HN. Not sure why. 'Big AI' didn't like it, perhaps?

pton_xd 3y ago
When an article crosses some threshhold of replies to upvotes per unit of time it gets deranked, presumably to discourage political / flame-war topics. Eg, this article still shows up on the second page. After a while, when the replies slow down, it will move back to the first page (if it's still getting upvotes).
- hermitcrab 3y ago
  Didn't know that. Thanks.

maaaaattttt 3y ago

The fact that internet is going to be filled with AI content is likely to happen. But for the rest I think it’s the classical bias science fiction has when predicting the future. More of the same. Or the linear transposition of the now (for a lack of better wording). Think CRT screens alongside flying cars. One could argue that LLMs have some form of intelligence already even though I believe it falls more under “very good intuition”. Even if the internet is filled with trash, we still have (and will continue to do so I hope) a lot of content of high quality available (all the books, podcasts, videos, etc. produced up until now). If you take human intelligence, it doesn’t require the whole internet to start to get smart. A lot of interactions with the real world and some good books/videos should be enough to get to AGI (and start the dreaded feedback loop). We just haven’t found a way to achieve it yet (AFAIK).

Eliah_Lakhin 3y ago

I think that people's believe in LLM is intelligence is closely tied to mistaken association between intelligence and the ability to speak. Parrots can speak, but they cannot reason. Moreover, evolutionary, birds learned to mimic the speech to fool other species. To fool in such way that other species would think that parrots are of the same specie. We, the humans, are smart enough to recognise that even though parrots can talk, they are not as smart as we are. Unfortunately for us, LLMs are much more advanced things than parrots, and they are capable to fool broad masses pretending that LLMs are of the human specie with all inherent features including the reasoning intelligence (that we cannot easily test externally). This is unfortunate, because such false believes slow down actual scientific progress towards the natural intelligence researches, and towards creating of the true reasoning artificial intelligence. Btw, I wouldn't be surprised that if one day the AGI created, it will not be able to speak at all, nor recognise the images.

krono 3y ago
I believe you are vastly underestimating the capabilities of parrots or similarly intelligent birds. At least when my pet parrot manages to fool me into whatever it is he wants to get out of me that time, he does it for his own primal benefit and not because it was programmed to adhere to some strict set of ethic and moral boundaries set by legal requirements and someone else's idea of how people should think and behave.
- smackeyacky 3y ago
  The smart behaviour of parrots and their speech synthesis are two different phenomenon. Parrots are definitely smart, and parrots can definitely speak, but they can't learn to speak beyond what they are trained to do. It's not like a parrot has ever spontaneously put together a coherent sentence independently. LLMs aren't even that smart. Not even close.
  - krono 3y ago
    Absolutely, I also replied to the OP's response below that I failed to specify that I was arguing parrots' ability to reason which I do believe exists to a great extend. Regardless, speech is just another means of communication. In the end it doesn't really matter if I use perfectly articulated Swahili or just scream "aaar" at you for a few seconds as long as I get the cheeseburger with extra cheese I want from you. These parrots, much like children, just push your buttons and are quite good at finding (or negotiating) the right pattern of things to do and sounds to make to get to a certain outcome.

lifeisstillgood 3y ago

The biggest threat here is not waves of bullshit, that’s nothing new. It’s knowing who to trust. One of the defining factors under the Stasi in East Germany, it was not that “ordinary people” could not recognise what they heard on the radio was bullshit, but that they could not know who to trust to say “that’s bullshit”. Every family has a Stasi informer, so whilst everyone knew the regieme was lying there was no critical mass event. There are a dozen technical solutions to “bullshit”. We just need to ensure we have institutions that support us standing up to it.

bitshiftfaced 3y ago

The assumption here is that future iterations of the technology require "BS"-riddled datasets. I question that assumption, both because the technology probably can improve solely with existing datasets and because we don't know that "synthetic" data isn't able to improve things.

sieste 3y ago

Isn't this really good news for artists? When everything you see is generic shit, it will be so much easier for truly creative thoughts, expressed through high quality writing, based on genuine experiences in the real world, to stand out. Also, this will put pressure on human creativity to produce weirder stuff that can't be produced by simply paraphrasing and remixing known ideas. Mass photography was bad news for the portrait painting industry, but forced painters to think outside known concepts and explore crazy abstract art. I totally look forward to whatever humans come up with next, that can't be easily generated by a computer program.

bulbosaur123 3y ago

SEO is dead. It will all be pay to play from now on. Want exposure? Open your wallet. Paid marketing will be the final bottleneck as problem of content has been resolved forever.

noduerme 3y ago

Ugh, the guy who thinks "prompt editor" is the job title of the future, and that people in every field will take classes on prompt editing. Gag. You know, there's no replacement for knowing how to do something yourself. And no, you cannot be as good creatively if you haven't learned a craft. (And no, prompt editing is not a craft any more than ordering food in a restaurant is cooking. Could you even describe in a useful way what was wrong with a soup you were served if you had never learned to cook?)

imacomputer 3y ago

This is a weird thread. I feel like there are many people who have no clue what they are talking about other than a few buzzwords, and make huge generalizations about AI or intelligence in general to _will their opinion into reality_. These statements are biased by either fear of loosing their job or fear of loosing on some investments on AI. Obviously, the enshittification threat as the author points out is significant. It will be interesting problem to deal with in the future, but until then I feel like no one knows when or what that will look like. At this rate, no one can predict what AI will bring us 6 months in the future even!

hermitcrab 3y ago

The argument that the problems of AI are going to be solved with MOAR AI, sounds suspiciously like the argument that the solution to America's gun problem will be solved with MOAR GUNS.

symbolicAGI 3y ago

Consider LLM code generation. It would an ideal domain for simulated data. For example, a version of popular software is released with new functions. A framework could simulate numerous plausible code snippets that exercise the new release, including edge cases. The code snippets would be amply commented, and proven to work in a test harness. This method is the opposite of depending upon the internet for training data. And this method is being used to train AI robots, that have no available internet training data.

evolextra 3y ago

[dead]

Luisvel 3y ago

Not sure why I expected something much more insighful when opening the link. The knowledge base that LLMs have today, is probably enough to derive very high quality content/knowledge. Data and information coming "from outside" won't cease to flow, but will otherwise increase. On the other hand, I see value in somehow indexing/marking human generated content, or at least human supervised/approved content.