15 ms·

Beware of AI pseudoscience and snake oil

237 points by bertman 3y ago

jablongo 3y ago
While he certainly points out things that have been true about AI hyperbole driven by profit seeking, its ironic how he himself is hyperbolic about the ineptitude of modern AIs while hawking his pdf e-book on the subject for $35.
bertman 3y ago
My favorite quote from the essay: "When they [AI companies] do ‘do’ science, they are still using science mostly as an aesthetic. They publish papers with grand claims but provide no access to any of the data or code used in the research. Their approach to science is what “my girlfriend goes to another school; you wouldn’t know her” is to high school cliques."
- version_five 3y ago
  That's like IBM doing research, or "thought leadership" from consulting companies. At the end of the day, it's sales / marketing content. It's a spectrum, and some of it is obviously high quality, bit a major aim is to draw in customers as opposed to directly supporting products or customer engagements. On the plus side, doing research does advance the state of the art, so it's one of the most productive marketing activities.
- williamcotton 3y ago
  Favorite, because it is hilariously wrong? Functioning code and tools are available for almost all of the relevant AI and ML work in academic publications. Yes, it took a big leak from Meta for a certain type of model, but the techniques to use LLMs and other transformer based tools are not vaporware. Most have code repositories and functioning open-source implementations. We can certainly be critical of the datasets used for testing but we can’t accuse people of selling snake oil when so many of us are implementing and gaining value from the techniques described in these academic publications.
  - latexr 3y ago
    > for almost all of the relevant AI and ML work The word “relevant” is an important qualifier which may be doing a lot of lifting. Who defines what is “relevant”? How do you interpret it for this particular case? I haven‘t looked at the matter myself, but the author did provide a source for their claim¹. Do you have any specific information on why it is wrong? It is not my intention to put you on the spot, I’m trying to understand if your point is based on data or gut feeling so I can myself get a better understanding of the validity of each argument. ¹ https://www.baldurbjarnason.com/2023/beware-of-ai-snake-oil/#fn:7 https://www.baldurbjarnason.com/2023/beware-of-ai-snake-oil/...
    - williamcotton 3y ago
      I’d say the two most relevant models right now are Stable Diffusion and GPT-4. Stable Diffusion is open in all senses of the word. GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. GPT-like things like LLaMA are, well, available. Direct, individually reproducible, these tools and their related academic publications qualify as computer science. Nullius in verba certainly applies.
      - latexr 3y ago
        > GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. Requiring a credit card to verify scientific claims is a large deterrent. And in addition one needs to provide them with a phone number, which many people will not be comfortable with when considering their CEO’s history.¹ Even ignoring the above, using the output of ChatGPT is definitely not the same as having “access to any of the data or code used in the research”, which is the author’s complaint and what the original commenter quoted. As for the other cases, we could only properly assess them if we discussed specific studies which I don’t think we’re going to do in the limited context of HN comments. Thank you for taking the time to explain your view. ¹ https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-...
- gumballindie 3y ago
  > They publish papers with grand claims but provide no access to any of the data or code used in the research. A common pattern around ai companies. To me it appears that a genuinely interesting technology has been hijacked by marketing and fud.
- synarchefriend 3y ago
  Providing no access to the underlying data is far more common in other fields of science than AI.
  - JohnFen 3y ago
    It is? In my experience with the research world, providing access to the underlying data is table stakes. If you don't do that, then nobody can verify the correctness of your papers.
- renewiltord 3y ago
  LOL this is such total bullshit. "Attention is All You Need" is from Google and is close to the inflection point of all this. This article is a complete clown show written in the style of online insult memes because that gets traction. It's by a clown masquerading as a technologist masquerading as a clown. See, I can do it too.
NoZebra120vClip 3y ago
I'm not sure about Watson's "expensive disaster", because all the public knows is "that charming supercomputer that starred on Jeopardy! and beat the pants off all other human contestants". That was a massive, enormous PR win for IBM and AI across the board. Nobody in the general public will forget that, and that will give a noticeable boost to any AI remotely like Watson, science be damned.
- SapporoChris 3y ago
  I had forgotten about Watson. I don't normally watch Jeopardy. However, maybe I'm not part of the general public. I did some fast checking, my numbers may be off, but it looks like Jeopardy has a viewership of around 11 million perhaps it was higher in 2011. Now news reports would have spread the word further, but that's a small percentage of the general public of the USA.
- HarHarVeryFunny 3y ago
  Sure, Watson+Jeopardy was good PR for IBM, as was DeepBlue, but even if the definition of success (vs disaster) is only PR-success, I'm not sure how lasting that is/was for IBM. Certainly from a technical POV Watson was just marketing hype - a common label applied to a grab-bag of unrelated unremarkable products. If you polled people on the street today about which companies they associate with AI, I doubt IBM would come out anywhere near the top. In fact, I'm not sure if you surveyed the under-20 crowd they could even tell you what IBM does, assuming they've even heard of them, other than perhaps being in the computer business in some vague way.
- kxyvr 3y ago
  It wasn't an expensive disaster for IBM, but it was for MD Anderson Cancer Center, which spent $62 million on a project using Watson that didn't work: https://www.medscape.com/viewarticle/876070 https://www.medscape.com/viewarticle/876070 Certainly, not all projects work and institutions need to take risk. However, there is a big financial and opportunity cost to this. Rather than spending $62 million on Watson, and assuming a $100k salary with a a cost multiplier of 2 ($200k for 1 FTE), that amount of money could have paid for 77.5 FTEs over the 4 year duration of the Watson contract (62e6/200e3/4). I suspect that the overall scientific contribution of those researchers would have been higher.
amelius 3y ago
Yes it's pseudoscience at least until someone comes up with a good metric to benchmark the quality of these LLMs. And then don't make the metric the target.
- davidthewatson 3y ago
  If you ask the LLMs, they'll tell you that reverse engineering a crawler's frontier from an LLM's data source is somewhere on the positive spectrum from possible to probable, while noting that it would take a lot of time, patience, and resources. Having worked in crawling for a few years and read the research, this strikes me as insightful in exactly the same way that AI presents the phenomenological insights of its human creators with no attribution. I've done enough experiments in the past year to have some insights about what was crawled and what was not. What we do not have is a DSM for AI, despite the fact that it mirrors the same self-report problems of its human creators. Sadly, we seem to have been unable to produce an AI without those problems - they permeate all of the current products in the space that I've tested. The entire discussion here is framed by an indictment of scientism on the one side and self-report on the other. The correct answers are elusive somewhere in the middle. Current SOTA in AI reminds me very much of the progression and disillusion from cybernetics and transpersonal psychology before and after the Dartmouth Workshop that created the field, respectively: https://en.wikipedia.org/wiki/Dartmouth_workshop https://en.wikipedia.org/wiki/Dartmouth_workshop
- PeterisP 3y ago
  IDK, we don't seem lacking in metrics. For measuring the quality of LLMs at the core task of language modeling (which then gets applied to other tasks), the classic information theory metric of perplexity works; for evaluating how well they transfer to other tasks, we can (and do) use metrics of those specific tasks, like the tasks in SuperGLUE dataset. All reasonable LLM papers do evaluation on these metrics, and they don't really "make the metric the target", you really can't do much in the large model pretraining to target SuperGLUE specifically.
blackbear_ 3y ago
In other news (https://archive.ph/WWKQ3 https://archive.ph/WWKQ3): > “I would advocate developing these types of AGI technologies in a cautious manner using the scientific method, where you try and do very careful controlled experiments to understand what the underlying system does,” Mr. Hassabis said. Myabe that's why Google was (seemingly) left in the dust?
- mach1ne 3y ago
  OpenAI minmaxed LLM development, DeepMind minmaxed other avenues of research, such as game-playing AI development. LLMs proved themselves first. Ockham’s Razor suggests there’s nothing more to it.
  - raincole 3y ago
    Yes. Today LLM seems like an "obvious" choice, but before ChatGPT, one could easily argue that game-playing AI is with much higher business value (to "play" stock market or even fighters), and chatbots are only good for customer service.
    - kriro 3y ago
      I'd argue both areas are successfull because they found good ways to solve the data problem and go "big data" and the use case while nice and shiney now is not as relevant. Big models trained on text work, because there's a lot of text availabe and the task of predicting the next tokens is nice because you get autolabeled data just by collecting a lot of text. Similarily RL solves this issue by simulations of sorts (supersimplified play a lot of games against eachother, judge somehow cleverly what was better, update stuff, rinse, repeat). I distinctly remember this lightbulb going off in my head when I first played with NLP. I came from ML for computer vision and trained ULMFiT on the entire Wikipedia and was thinking...oh neat no labeling that's magical, if I had more compute I could basically feed this infinite data. I'm not following synthetic data closely (I've used it to train models before) but I see a bright future for this space (like game engines used to simulate big worlds simply to generate data or training in said worlds and transfering).
- 3y ago
Moneyyyyy 3y ago
Plenty of good science is out there. And yes the current stage gives us a very very clear path ahead. There is also now gpt4 model available on openai which has access to python and can run it. We are definitely going in the direction of LLM combining with GAN ideas. Googles med ml (I think it's med-palm-ml)model is progressing too. This hype is not just a hype it's the unknotting of the ml knot. We know for a while that plenty of actors do stuff with ml but it wasn't as good and as obvious to people like now that ml is super crazy critical.
somewhereoutth 3y ago
I think the term 'Intelligence' is what is causing the problem. If something like 'statistical modelling' or 'quasi-stochastic information compression' was used instead there would be much less hot air. For example, in Real Time Analytics, there is something called a 'sketch', which though clever statistical techniques allows approximations of such things like top 10, average etc, without having to recalculate from the whole dataset each time of asking. Nobody claims this has anything to do with intelligence, artifical or otherwise.
- sfpotter 3y ago
  My impression is that the vast majority of people who are currently engaged in the “discourse” surrounding AI do not have the technical background to make this distinction. In my opinion, there’s an element of “the man behind the curtain” which is being unethically leveraged by AI companies to drive sales and hype.
  - shanebellone 3y ago
    Plus, there seems to be a chronic conflation of "capable of configuring" and "fundamentally understanding the technology".
  - yowlingcat 3y ago
    I agree and there is an enormous moral hazard to the latter. Just looking at what Hinton has been discussing post G, and you see ideological support for a narrow model, shameless anthropomorphization of LLM behavior, and downplaying of "hallucinations" (which I'd rather simply call what it is, which is lying). The moral hazard (for sure not new) is that those most equipped to call out the overplaying of the technology's abilities are also those most personally incented (via fame for their creation), or commercially incented (as you mentioned) to drive sales and hype.
    - HybridCurve 3y ago
      > which I'd rather simply call what it is, which is lying I like the term "bullshitting" as it implies it acts like it knows, but doesn't. I can't stand it when they call it "hallucinating", as it instead might imply the problem is with the interpretation of the sensory input. This term might be more appropriate to describe adversarial attacks executed through information input.
DannyBee 3y ago
The Dutch system cited didn't use AI and is 15+ years old (ie predates current AI hype by far). Most AI papers have working code available. Etc This entire article is rife with trivial to find errors that totally invalidate it's claims. there may be a problem, there may not, this article doesn’t help figure it out either way. It feels like narrative was written first, then someone went looking to try to find newsbites that sound like they support it.
- latexr 3y ago
  I see plenty of links to recent papers (last two years). > This entire article is rife with trivial to find errors that totally invalidate it's claims. Could you share some? Maybe they’re trivial for you but handwaving it doesn’t help others identify what is correct or not. For your claim to be true either the multiple papers cited would have to contain glaring flaws or the author would have to be wildly misinterpreting them. That doesn’t seem trivial to assess. > It feels like narrative was written first, then someone went looking to try to find newsbites that sound like they support it. From the author’s other writings, the opposite seems to be true: https://www.baldurbjarnason.com/2023/thank-you-for-your-patience/ https://www.baldurbjarnason.com/2023/thank-you-for-your-pati... https://www.baldurbjarnason.com/2023/why-i-wrote-the-intelligence-illusion/ https://www.baldurbjarnason.com/2023/why-i-wrote-the-intelli...
  - UncleEntity 3y ago
    A super obvious one is it’s memorizing all the test answers so the results are flawed. Simple solution: come up with new questions. Plus no knowledge test I’ve ever taken in my life expected novel solutions but were merely testing how well you understand the subject.
- nottorp 3y ago
  > The Dutch system cited didn't use AI Expert systems aren't AI? Decision trees aren't AI? Only neural networks (which incidentally have been defined 50 years ago) are AI now?
kalkin 3y ago
The article conflates AGI with "consciousness", which is a really basic misunderstanding. Pretty discrediting.
- hartator 3y ago
  How is it not?
  - pookha 3y ago
    Most AI uses some variation of deep-learning (CNN-resnet for example). In my mind that's really just some variation of fancy counting.
  - kalkin 3y ago
    Consciousness is about experience. Intelligence is about capability. Dogs are conscious, but can't add two digit numbers. ChatGPT is not conscious (almost certainly, although we don't really understand consciousness well enough to prove this), but can add two digit numbers.
    - Symmetry 3y ago
      It's not necessarily about experience except in the discipline of Philosophy. Cognitive scientists have their own ideas and theories about consciousness, like Global Workspace Theory. However, since LLMs are feed forward networks incapable for remembering things beyond their input window they also conspicuously fail to be conscious by those definitions too: [1]https://en.wikipedia.org/wiki/Global_workspace_theory https://en.wikipedia.org/wiki/Global_workspace_theory
  - goatlover 3y ago
    Consciousness is subjective experience. Intelligence is problem solving, abstracting, reasoning, generalizing. There's no requirement for those things to be accompanied with subjectivity. And there's no reason to think LLMs have subjective experiences.
mmaunder 3y ago
Marketing claims have always been problematic, in every industry. But you can’t argue with results.
- deleted 3y ago
  [deleted]
- hammyhavoc 3y ago
  Well, you can if the results are objectively unreliable, or the technology is wildly inappropriate in a workflow versus conditional logic where x always does y. Some of the suggested uses are as absurd as blockchain mania trying to replace databases.
drakmo 3y ago
One thing is for sure. His book will sell like hotcakes. The recipe to this is as follwoing: - 50 oz of fear as our base ingredient - mix in some grains of truth (like OpenAI being a company very restrained with their IP and competitional advantage but painting a different picture in the public) - stuff it up with halftruths, distractions and handpicked cherries as you like to make it look colorful and shiny (nevermind how these taste because if they taste it they already bought it and you can never go completely wrong in taste if your base is sugar aka fear)
GaggiX 3y ago
Honestly, I rarely see research as accessible as in the field of AI: a new method is published to fine-tune SD, dozens of repos reimplement the technique and improve it and this happens every day.
lincpa 3y ago
[dead]
sebastianconcpt 3y ago
I'm afraid this isn't a problem exclusive to AI. There is a kind of intellectual fraud that is being normalized sacrificing truth in the altar of tech hype. AI being crucial for the escalated propaganda wars isn't exactly helping to alleviate this issue.
- the_snooze 3y ago
  >There is a kind of intellectual fraud that is being normalized sacrificing truth in the altar of tech hype. The smell test I use to sniff those out involve asking two questions: Who is this tech for? And what are its limitations? If the answer is a hyped "everyone, no limits!" then it's almost certainly tech woo-woo.
Garlef 3y ago
To me it seems that demanding scientific rigour misses the mark. What will be important in the end is results. I think it's pretty obvious that image generation such as Midjourney and Stable Diffusion is an incredible technological advancement. Who cares if the science behind it is solid? Or even if the papers published by companies that sell the technology are solid? LLMs still need to prove themselves to some extent but it's already clear that something unprecedented is actually going on here. That's why everyone is hyped/freaked out. But we have not yet figured out exactly how useful the technology really is.
- bannedbybros 3y ago
  [dead]
smcleod 3y ago
I think the author is far too pessimistic about the capabilities of ML. I think we're already beyond the "it might not live up to the hype" stage and well into - "holy moley this is actually really well (for certain things) already - now let's figure out how to use and iterate upon it" stage. However - the level of Silicon Valley Bro VC funded hype is totally out of this world. Many of the claims of 'science' being used in marketing of many services seem to be misleading a lot of people as to how it all works. Some might even jest that they're far closer to magic than science as far as our understanding goes. Coupled with the very serious risks (not that we can 'fix' that now - the cat is out of the bag), how slow governments are when it comes to technology (not to mention the seriously lacking amount of regulation in the American tech sector), politicians with little-to-no technical understanding in general let alone of ML when combined with the corporations that already have market dominance in many areas - makes for an interesting time for certain. Marketing departments and now the general public using the term AI instead of ML is just the tip of the iceberg.
- packetlost 3y ago
  The practical applications of AI is undoubtedly overblown, but I do agree that it's ability to generate content is impressive and will (or rather, already is) being used for practical things.
- fwungy 3y ago
  The first 80% of AI tasks will be easy enough. The next 20% will be increasingly difficult with the last 1% virtually impossible. Same with driverless cars.
  - danielmarkbruce 3y ago
    Folks who are pessimistic on driverless cars appear to move the goals on the last 1% repeatedly. Self driving cars are driving themselves around Phoenix and SF right at this minute.
    - vajrabum 3y ago
      I passed a driverless car from Cruise in San Francisco on my commute just the other day.
jhbadger 3y ago
>AI research is poor. Many of their claims will be proven wrong. This is true of every field of science. As Dara Ó Brian memorably said, "Science knows it doesn't know everything. If it did, it would stop".
MichaelMoser123 3y ago
now is 'AGI is just around the corner' an example of techno-optimism or techno-pessimism?
- trashtester 3y ago
  Both, and it will positively cause humanity's extinction.
  - MichaelMoser123 3y ago
    not sure about that. i suspect that AGI might be a tool to help one of the bigcorps to grab power. Until now the state and the bigcorps were keeping each other in check, there was a kind of 'balance of power'. Now they might get a tool to upset this balance of power. Whoever gets AGI first might turn into Buy-N-Large or ACME Corp, whatever... (i suspect it's all about the big price - ultimate power) - it's kind of the 'ring to rule them all' - if AGI is achievable within a realistic time frame... * now let's see if that's the dystopian future that we are really after :-) *
azubinski 3y ago
"To forestall having a “dasvidaniya comrade” moment where a self-aware GPT-5 shoves an icepick into their ear, Trotsky-style, they put together a ‘red team’ that tested whether GPT-4 was capable of ‘escaping’ or turning on its masters in some way" You can agree with the author, you can disagree with the author, but he has a Style.
hospitalJail 3y ago
The author never used ChatGPT. Its clear from their writing. They would be able to acknowledge the risks, but they pretended it was impossible because the quality isnt perfect. They don't know how to use the tool/never used the tool, but decided to write a book on it? This seems like a trope: "LLMs suck! No I never tried it before"
- alostpuppy 3y ago
  Alternative pitch… the author used ChatGPT to write this.
agentultra 3y ago
A thousand times this. It's unfortunately too easy to find medieval thinking in tech and computer science circles. Many who are ready and eager to believe rather than remain skeptical. A great number of people who claim the title of researcher or engineer that can only use fractured metaphors and tenuous analogies to extrapolate science fictional predictions from marketing copy. And a great deal of them have plenty of financial incentive to believe and convince others to do so. It was disheartening to see so many in the tech community get taken in by the crypto-craze when all manner of inevitable futures were imagined for us. LLM's are a tool. Data and algorithms. To attribute self awareness and intelligence to them is pure hyperbole.
- kalkin 3y ago
  Self-awareness and intelligence (in the sense in which AI has any and will likely have more in five years) are not the same thing. My cats have more self awareness than ChatGPT but the latter can do a lot of intellectual tasks that nonhuman animals can't. What's disheartening for me is to see people make confident pronouncements about how other people are being taken in by scifi and hype and there's nothing to worry about even while displaying what seem like basic conceptual confusions.
  - agentultra 3y ago
    I'm not worried that GPT-4 is going to "escape its jail, become self aware, and start building a robot army to conquer the world." Or whatever. I wasn't worried that the future of financial markets and payment systems were going to be replaced with crypto-currencies and the IMF replaced with the Ethereum developers. I didn't believe for a second that they would successfully supplant the financial institutions and state governments and escape regulation. We were also supposed to see the modern Internet replaced with web3. It was inevitable! These were all hyperbolic claims made by crypto enthusiasts for years. They were preposterous from day one but it didn't stop many smart, intelligent people from being taken in and converted to believers. There are real dangers! However I think those dangers come from other humans and human institutions. Just as the dangers of crypto came from real humans pulling off huge scams and fraud. Whether it's corporations hyping their AI products to health care professionals and causing thousands of misdiagnoses that lead to chronic health problems or death; to scammers using them to trick family members of their targets to divulge useful information... that's all humans being humans and leveraging a tool that makes it easier for them to cause harm (intentionally or otherwise). What we need are people cutting through the hype so that regular people don't mistake negligence or intentional malice with the "sentience" of "AGI."
xwdv 3y ago
The common man’s perception of AI and its capabilities right now is similar to how people would look at you like you’re the Wizard of Oz just by building a web page back in 1999, and if you were young enough they would even herald you as “the next Bill Gates!”
mountainofdeath 3y ago
"Criti-hype", I like that word. It indeed is like car salesperson hyping that the very thing they are selling is almost not street legal, begging the buyer to try it on and placing that seed of thought. On another note, everyone is now using "algorithms and machine learning" in the same way they used "blockchain" in 2021. The best part is, the majority of the "algorithms" being used are basically glorified Excel models; something you would expect a fresher in college to piece together for their Physics 101 class.
robofanatic 3y ago
These days every software company has AI and ML acronym in their tag lines irrespective of whether or not they have anything to do with it.
jschveibinz 3y ago
We continue to see these “oh it’s not…” and “watch out for…” articles. Let’s take a step back and just take it for what it is. If I walk up to a regular someone on the street and say “please write me an outline for a short story about tuna fish and bunny rabbits,” you’re going to get a blank stare. Heck, even an experienced story teller might have a tough time with that. But these current AI tools will give you a credible outline for that bunny-tuna story in about 10 seconds. It’s not the best story. But it’s a good starting point. To me, that’s freaking great. Maybe it’s a little esoteric, but still amazing. And there are a zillion other things that these tools can produce very quickly that are excellent starting points. Humans can’t do this. Period. There is no snake oil. The tools have instant access to decades of human development. Some of us can’t find Washington, DC on a map let alone write a short story outline or 100 lines of code in 10 seconds. How does that “talent” translate to productivity, i.e. useful output? That’s up to the users. Some people will know how to ask the right questions. Other people won’t. There will definitely be another “sorting” of people based on access and ability to use the tools. But this is the course of human development. It’s evolutionary, really. From now on we go forward with these tools. That’s my take.
- sangnoir 3y ago
  > If I walk up to a regular someone on the street and say “please write me an outline for a short story about tuna fish and bunny rabbits,” you’re going to get a blank stare. Probably they'd be trying to assess if you're sane and not a danger to them. This is a task a 6 year-old could accomplish with style and vigor.
- anotherman554 3y ago
  "But these current AI tools will give you a credible outline for that bunny-tuna story in about 10 seconds. It’s not the best story. But it’s a good starting point." A good starting point for what? If the end result is a story that nobody but yourself will ever want to read, why not simply copy and paste "All work and no play makes Johnny a dull boy" a thousand times, no need for an A.I.
greenhearth 3y ago
The main theme of the article for me is the discrepancies between marketing (shiny cable news segments and social media) and real, dirty-hands research and development. They are completely out of sync. I am studying Comp Ling and one of the first understandings you develop is that Natural Language Processing through neural networks can achieve a level of accuracy and efficiency (which is pretty high), but eventual limitations occur where improvement is impossible. And perfection is certainly an impossibility.
helen___keller 3y ago
I never found a novel useful product after many hours of scouring web3 startups. Usually it was “X for web3”, predicated on something else useful existing to justify the web3 ecosystem I’m cautiously more optimistic about AI (this time) since we’ve seen impressive functionality from OpenAI and such, and the availability of an impressive LLM as an API excites possibilities. In my head a large open question is how flexible / powerful fine tuning and context/prompt engineering can be; if it’s not powerful enough, then I predict a wave of snake oil failures, and the space fizzles out for a few more years. On the other hand, if you can take an API, some fine tuning data, and some prompt engineers, and build an actually useful service from that, then the space is about to explode with new unicorns
scp3125 3y ago
This is a tangent, but snake oil was medicinally effective. The colloquialism "snake oil salesman" comes from the old scam of confidence men selling a cheap, adulterated (and sometimes drugged) mineral oil as snake oil (based on the Chinese water snake from Chinese traditional medicine). People would pay a premium for what they thought was medicine, only to find out later that the confidence men bamboozled them and skipped town with their money. https://en.wikipedia.org/wiki/Snake_oil https://en.wikipedia.org/wiki/Snake_oil
- iudqnolq 3y ago
  From your source: > In 18th-century Europe, especially in the UK, viper oil had been commonly recommended for many afflictions, including the ones for which oil from the rattlesnake (pit viper), a type of viper native to America, was subsequently favored to treat rheumatism and skin diseases.[6] Though there are accounts of oil obtained from the fat of various vipers in the Western world, the claims of its effectiveness as a medicine have never been thoroughly examined, and its efficacy is unknown. (emphasis added)
kriro 3y ago
We might be going full circle soon and in need of "Beware of Beware of AI pseudoscience and snake oil". In my personal bubble there's roughly the same number of AI snake oil stuff popping up as there is finger wiggly "be weary off AI claims" stuff that lacks even the simplest understanding of AI let alone modern things like how transformers actually work.
nottorp 3y ago
Crypto in Rust (on HN and maybe other techie places) AI Anyone want to make predictions on what the next overhyped topic will be in a few years? Maybe we should ask ChatGPT :)
SamPatt 3y ago
I feel like the article would have been a lot more convincing in the pre-GPT era. I use GPT-4 daily, and the author is simply wrong about its capabilities.
gtirloni 3y ago
This showed up in the front page and was quickly downvoted to the second page before I had finished the first page. The Kool Aid is strong today.
- daveguy 3y ago
  You can't downvote a whole thread, only comments. Threads can only be flagged or upvoted and the thread is not flagged. If there isn't enough interest it fades to the second page after a while. That's what happened here.
  - gtirloni 3y ago
    Thanks for the clarification. Does it penalize a submission when someone flags it (but it doesn't reach whatever threshold there is to actually mark it as flagged)? I'm assuming there's a threshold.
    - daveguy 3y ago
      That's a good question and it looks like a flag is considered in ranking before it goes defunct. From the faq [0]: How are stories ranked? ...upvotes [positive]...time [negative]... Other factors affecting rank include user flags, anti-abuse software, software which demotes overheated discussions, account or site weighting, and moderator action. So for all intents and purposes it looks like a flag acts in some way as a downvote. I was just confused by the term "downvote". They don't have the same effect as an upvote (maybe more effect?) but they do drop a story in the rankings even though they may not be removed. Those last three factors look like it can be rather arbitrary. But in general I prefer the quality of articles and discussions here to other sites.
Sloppy 3y ago
Hype is ever-present, beware. OK. ML was overhyped but has proven useful, OK. But the new Generative AIs are fundamentally different. They are trained without any notion of a ground truth. They therefore have no concept of reality. They only know what is real-ish. They are psychotic. Temper the hype with the caveat--"but what it produces is often untruthful or unreal" and the hype is instantly clear. This essay fails to account for why GAI is so different. This problem can only be solved with new methods for training that have as yet not been invented. Older style ML is based on using some ground truth and comes with a metric that measures how close it is to this truth. Not so of any of the new GAI, so continue to beware until researcher figure out how to measure truth--and good luck with that.
RHSman2 3y ago
The pursuit of AI is intellectually interesting however we aren’t intellectually advanced enough to see why it is a very bad idea.
wanderingmind 3y ago
You mean like most so called regular science which is never reproducible as well
kaycey2022 3y ago
This is an article that is both timely and necessary. Way too many people are simply leaning into the AI hype and there is simply not enough scepticism to go with it. I refer to one of the comments made in the thread about the state of self driving cars: “People who haven’t been able to make any progress in self driving cars are now claiming that full conscious AI is just an year away!”

237 points by bertman 3y ago

While he certainly points out things that have been true about AI hyperbole driven by profit seeking, its ironic how he himself is hyperbolic about the ineptitude of modern AIs while hawking his pdf e-book on the subject for $35.

bertman 3y ago

My favorite quote from the essay: "When they [AI companies] do ‘do’ science, they are still using science mostly as an aesthetic. They publish papers with grand claims but provide no access to any of the data or code used in the research. Their approach to science is what “my girlfriend goes to another school; you wouldn’t know her” is to high school cliques."

version_five 3y ago
That's like IBM doing research, or "thought leadership" from consulting companies. At the end of the day, it's sales / marketing content. It's a spectrum, and some of it is obviously high quality, bit a major aim is to draw in customers as opposed to directly supporting products or customer engagements. On the plus side, doing research does advance the state of the art, so it's one of the most productive marketing activities.
williamcotton 3y ago
Favorite, because it is hilariously wrong? Functioning code and tools are available for almost all of the relevant AI and ML work in academic publications. Yes, it took a big leak from Meta for a certain type of model, but the techniques to use LLMs and other transformer based tools are not vaporware. Most have code repositories and functioning open-source implementations. We can certainly be critical of the datasets used for testing but we can’t accuse people of selling snake oil when so many of us are implementing and gaining value from the techniques described in these academic publications.
- latexr 3y ago
  > for almost all of the relevant AI and ML work The word “relevant” is an important qualifier which may be doing a lot of lifting. Who defines what is “relevant”? How do you interpret it for this particular case? I haven‘t looked at the matter myself, but the author did provide a source for their claim¹. Do you have any specific information on why it is wrong? It is not my intention to put you on the spot, I’m trying to understand if your point is based on data or gut feeling so I can myself get a better understanding of the validity of each argument. ¹ https://www.baldurbjarnason.com/2023/beware-of-ai-snake-oil/#fn:7 https://www.baldurbjarnason.com/2023/beware-of-ai-snake-oil/...
  - williamcotton 3y ago
    I’d say the two most relevant models right now are Stable Diffusion and GPT-4. Stable Diffusion is open in all senses of the word. GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. GPT-like things like LLaMA are, well, available. Direct, individually reproducible, these tools and their related academic publications qualify as computer science. Nullius in verba certainly applies.
    - latexr 3y ago
      > GPT-4 is closed but the techniques it enables are reproducible by anyone with a credit card. Requiring a credit card to verify scientific claims is a large deterrent. And in addition one needs to provide them with a phone number, which many people will not be comfortable with when considering their CEO’s history.¹ Even ignoring the above, using the output of ChatGPT is definitely not the same as having “access to any of the data or code used in the research”, which is the author’s complaint and what the original commenter quoted. As for the other cases, we could only properly assess them if we discussed specific studies which I don’t think we’re going to do in the limited context of HN comments. Thank you for taking the time to explain your view. ¹ https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-...
gumballindie 3y ago
> They publish papers with grand claims but provide no access to any of the data or code used in the research. A common pattern around ai companies. To me it appears that a genuinely interesting technology has been hijacked by marketing and fud.
synarchefriend 3y ago
Providing no access to the underlying data is far more common in other fields of science than AI.
- JohnFen 3y ago
  It is? In my experience with the research world, providing access to the underlying data is table stakes. If you don't do that, then nobody can verify the correctness of your papers.
renewiltord 3y ago
LOL this is such total bullshit. "Attention is All You Need" is from Google and is close to the inflection point of all this. This article is a complete clown show written in the style of online insult memes because that gets traction. It's by a clown masquerading as a technologist masquerading as a clown. See, I can do it too.

NoZebra120vClip 3y ago

I'm not sure about Watson's "expensive disaster", because all the public knows is "that charming supercomputer that starred on Jeopardy! and beat the pants off all other human contestants". That was a massive, enormous PR win for IBM and AI across the board. Nobody in the general public will forget that, and that will give a noticeable boost to any AI remotely like Watson, science be damned.

SapporoChris 3y ago
I had forgotten about Watson. I don't normally watch Jeopardy. However, maybe I'm not part of the general public. I did some fast checking, my numbers may be off, but it looks like Jeopardy has a viewership of around 11 million perhaps it was higher in 2011. Now news reports would have spread the word further, but that's a small percentage of the general public of the USA.
HarHarVeryFunny 3y ago
Sure, Watson+Jeopardy was good PR for IBM, as was DeepBlue, but even if the definition of success (vs disaster) is only PR-success, I'm not sure how lasting that is/was for IBM. Certainly from a technical POV Watson was just marketing hype - a common label applied to a grab-bag of unrelated unremarkable products. If you polled people on the street today about which companies they associate with AI, I doubt IBM would come out anywhere near the top. In fact, I'm not sure if you surveyed the under-20 crowd they could even tell you what IBM does, assuming they've even heard of them, other than perhaps being in the computer business in some vague way.
kxyvr 3y ago
It wasn't an expensive disaster for IBM, but it was for MD Anderson Cancer Center, which spent $62 million on a project using Watson that didn't work: https://www.medscape.com/viewarticle/876070 https://www.medscape.com/viewarticle/876070 Certainly, not all projects work and institutions need to take risk. However, there is a big financial and opportunity cost to this. Rather than spending $62 million on Watson, and assuming a $100k salary with a a cost multiplier of 2 ($200k for 1 FTE), that amount of money could have paid for 77.5 FTEs over the 4 year duration of the Watson contract (62e6/200e3/4). I suspect that the overall scientific contribution of those researchers would have been higher.

amelius 3y ago

Yes it's pseudoscience at least until someone comes up with a good metric to benchmark the quality of these LLMs. And then don't make the metric the target.

davidthewatson 3y ago
If you ask the LLMs, they'll tell you that reverse engineering a crawler's frontier from an LLM's data source is somewhere on the positive spectrum from possible to probable, while noting that it would take a lot of time, patience, and resources. Having worked in crawling for a few years and read the research, this strikes me as insightful in exactly the same way that AI presents the phenomenological insights of its human creators with no attribution. I've done enough experiments in the past year to have some insights about what was crawled and what was not. What we do not have is a DSM for AI, despite the fact that it mirrors the same self-report problems of its human creators. Sadly, we seem to have been unable to produce an AI without those problems - they permeate all of the current products in the space that I've tested. The entire discussion here is framed by an indictment of scientism on the one side and self-report on the other. The correct answers are elusive somewhere in the middle. Current SOTA in AI reminds me very much of the progression and disillusion from cybernetics and transpersonal psychology before and after the Dartmouth Workshop that created the field, respectively: https://en.wikipedia.org/wiki/Dartmouth_workshop https://en.wikipedia.org/wiki/Dartmouth_workshop
PeterisP 3y ago
IDK, we don't seem lacking in metrics. For measuring the quality of LLMs at the core task of language modeling (which then gets applied to other tasks), the classic information theory metric of perplexity works; for evaluating how well they transfer to other tasks, we can (and do) use metrics of those specific tasks, like the tasks in SuperGLUE dataset. All reasonable LLM papers do evaluation on these metrics, and they don't really "make the metric the target", you really can't do much in the large model pretraining to target SuperGLUE specifically.

blackbear_ 3y ago

In other news (https://archive.ph/WWKQ3 https://archive.ph/WWKQ3): > “I would advocate developing these types of AGI technologies in a cautious manner using the scientific method, where you try and do very careful controlled experiments to understand what the underlying system does,” Mr. Hassabis said. Myabe that's why Google was (seemingly) left in the dust?

mach1ne 3y ago
OpenAI minmaxed LLM development, DeepMind minmaxed other avenues of research, such as game-playing AI development. LLMs proved themselves first. Ockham’s Razor suggests there’s nothing more to it.
- raincole 3y ago
  Yes. Today LLM seems like an "obvious" choice, but before ChatGPT, one could easily argue that game-playing AI is with much higher business value (to "play" stock market or even fighters), and chatbots are only good for customer service.
  - kriro 3y ago
    I'd argue both areas are successfull because they found good ways to solve the data problem and go "big data" and the use case while nice and shiney now is not as relevant. Big models trained on text work, because there's a lot of text availabe and the task of predicting the next tokens is nice because you get autolabeled data just by collecting a lot of text. Similarily RL solves this issue by simulations of sorts (supersimplified play a lot of games against eachother, judge somehow cleverly what was better, update stuff, rinse, repeat). I distinctly remember this lightbulb going off in my head when I first played with NLP. I came from ML for computer vision and trained ULMFiT on the entire Wikipedia and was thinking...oh neat no labeling that's magical, if I had more compute I could basically feed this infinite data. I'm not following synthetic data closely (I've used it to train models before) but I see a bright future for this space (like game engines used to simulate big worlds simply to generate data or training in said worlds and transfering).
3y ago

Moneyyyyy 3y ago

Plenty of good science is out there. And yes the current stage gives us a very very clear path ahead. There is also now gpt4 model available on openai which has access to python and can run it. We are definitely going in the direction of LLM combining with GAN ideas. Googles med ml (I think it's med-palm-ml)model is progressing too. This hype is not just a hype it's the unknotting of the ml knot. We know for a while that plenty of actors do stuff with ml but it wasn't as good and as obvious to people like now that ml is super crazy critical.

somewhereoutth 3y ago

I think the term 'Intelligence' is what is causing the problem. If something like 'statistical modelling' or 'quasi-stochastic information compression' was used instead there would be much less hot air. For example, in Real Time Analytics, there is something called a 'sketch', which though clever statistical techniques allows approximations of such things like top 10, average etc, without having to recalculate from the whole dataset each time of asking. Nobody claims this has anything to do with intelligence, artifical or otherwise.

sfpotter 3y ago
My impression is that the vast majority of people who are currently engaged in the “discourse” surrounding AI do not have the technical background to make this distinction. In my opinion, there’s an element of “the man behind the curtain” which is being unethically leveraged by AI companies to drive sales and hype.
- shanebellone 3y ago
  Plus, there seems to be a chronic conflation of "capable of configuring" and "fundamentally understanding the technology".
- yowlingcat 3y ago
  I agree and there is an enormous moral hazard to the latter. Just looking at what Hinton has been discussing post G, and you see ideological support for a narrow model, shameless anthropomorphization of LLM behavior, and downplaying of "hallucinations" (which I'd rather simply call what it is, which is lying). The moral hazard (for sure not new) is that those most equipped to call out the overplaying of the technology's abilities are also those most personally incented (via fame for their creation), or commercially incented (as you mentioned) to drive sales and hype.
  - HybridCurve 3y ago
    > which I'd rather simply call what it is, which is lying I like the term "bullshitting" as it implies it acts like it knows, but doesn't. I can't stand it when they call it "hallucinating", as it instead might imply the problem is with the interpretation of the sensory input. This term might be more appropriate to describe adversarial attacks executed through information input.

DannyBee 3y ago

The Dutch system cited didn't use AI and is 15+ years old (ie predates current AI hype by far). Most AI papers have working code available. Etc This entire article is rife with trivial to find errors that totally invalidate it's claims. there may be a problem, there may not, this article doesn’t help figure it out either way. It feels like narrative was written first, then someone went looking to try to find newsbites that sound like they support it.

latexr 3y ago
I see plenty of links to recent papers (last two years). > This entire article is rife with trivial to find errors that totally invalidate it's claims. Could you share some? Maybe they’re trivial for you but handwaving it doesn’t help others identify what is correct or not. For your claim to be true either the multiple papers cited would have to contain glaring flaws or the author would have to be wildly misinterpreting them. That doesn’t seem trivial to assess. > It feels like narrative was written first, then someone went looking to try to find newsbites that sound like they support it. From the author’s other writings, the opposite seems to be true: https://www.baldurbjarnason.com/2023/thank-you-for-your-patience/ https://www.baldurbjarnason.com/2023/thank-you-for-your-pati... https://www.baldurbjarnason.com/2023/why-i-wrote-the-intelligence-illusion/ https://www.baldurbjarnason.com/2023/why-i-wrote-the-intelli...
- UncleEntity 3y ago
  A super obvious one is it’s memorizing all the test answers so the results are flawed. Simple solution: come up with new questions. Plus no knowledge test I’ve ever taken in my life expected novel solutions but were merely testing how well you understand the subject.
nottorp 3y ago
> The Dutch system cited didn't use AI Expert systems aren't AI? Decision trees aren't AI? Only neural networks (which incidentally have been defined 50 years ago) are AI now?

kalkin 3y ago

The article conflates AGI with "consciousness", which is a really basic misunderstanding. Pretty discrediting.

hartator 3y ago
How is it not?
- pookha 3y ago
  Most AI uses some variation of deep-learning (CNN-resnet for example). In my mind that's really just some variation of fancy counting.
- kalkin 3y ago
  Consciousness is about experience. Intelligence is about capability. Dogs are conscious, but can't add two digit numbers. ChatGPT is not conscious (almost certainly, although we don't really understand consciousness well enough to prove this), but can add two digit numbers.
  - Symmetry 3y ago
    It's not necessarily about experience except in the discipline of Philosophy. Cognitive scientists have their own ideas and theories about consciousness, like Global Workspace Theory. However, since LLMs are feed forward networks incapable for remembering things beyond their input window they also conspicuously fail to be conscious by those definitions too: [1]https://en.wikipedia.org/wiki/Global_workspace_theory https://en.wikipedia.org/wiki/Global_workspace_theory
- goatlover 3y ago
  Consciousness is subjective experience. Intelligence is problem solving, abstracting, reasoning, generalizing. There's no requirement for those things to be accompanied with subjectivity. And there's no reason to think LLMs have subjective experiences.

mmaunder 3y ago

Marketing claims have always been problematic, in every industry. But you can’t argue with results.

deleted 3y ago
[deleted]
hammyhavoc 3y ago
Well, you can if the results are objectively unreliable, or the technology is wildly inappropriate in a workflow versus conditional logic where x always does y. Some of the suggested uses are as absurd as blockchain mania trying to replace databases.

drakmo 3y ago

One thing is for sure. His book will sell like hotcakes. The recipe to this is as follwoing: - 50 oz of fear as our base ingredient - mix in some grains of truth (like OpenAI being a company very restrained with their IP and competitional advantage but painting a different picture in the public) - stuff it up with halftruths, distractions and handpicked cherries as you like to make it look colorful and shiny (nevermind how these taste because if they taste it they already bought it and you can never go completely wrong in taste if your base is sugar aka fear)

GaggiX 3y ago

Honestly, I rarely see research as accessible as in the field of AI: a new method is published to fine-tune SD, dozens of repos reimplement the technique and improve it and this happens every day.

lincpa 3y ago

[dead]

sebastianconcpt 3y ago

I'm afraid this isn't a problem exclusive to AI. There is a kind of intellectual fraud that is being normalized sacrificing truth in the altar of tech hype. AI being crucial for the escalated propaganda wars isn't exactly helping to alleviate this issue.

the_snooze 3y ago
>There is a kind of intellectual fraud that is being normalized sacrificing truth in the altar of tech hype. The smell test I use to sniff those out involve asking two questions: Who is this tech for? And what are its limitations? If the answer is a hyped "everyone, no limits!" then it's almost certainly tech woo-woo.

Garlef 3y ago

To me it seems that demanding scientific rigour misses the mark. What will be important in the end is results. I think it's pretty obvious that image generation such as Midjourney and Stable Diffusion is an incredible technological advancement. Who cares if the science behind it is solid? Or even if the papers published by companies that sell the technology are solid? LLMs still need to prove themselves to some extent but it's already clear that something unprecedented is actually going on here. That's why everyone is hyped/freaked out. But we have not yet figured out exactly how useful the technology really is.

bannedbybros 3y ago
[dead]

smcleod 3y ago

I think the author is far too pessimistic about the capabilities of ML. I think we're already beyond the "it might not live up to the hype" stage and well into - "holy moley this is actually really well (for certain things) already - now let's figure out how to use and iterate upon it" stage. However - the level of Silicon Valley Bro VC funded hype is totally out of this world. Many of the claims of 'science' being used in marketing of many services seem to be misleading a lot of people as to how it all works. Some might even jest that they're far closer to magic than science as far as our understanding goes. Coupled with the very serious risks (not that we can 'fix' that now - the cat is out of the bag), how slow governments are when it comes to technology (not to mention the seriously lacking amount of regulation in the American tech sector), politicians with little-to-no technical understanding in general let alone of ML when combined with the corporations that already have market dominance in many areas - makes for an interesting time for certain. Marketing departments and now the general public using the term AI instead of ML is just the tip of the iceberg.

packetlost 3y ago
The practical applications of AI is undoubtedly overblown, but I do agree that it's ability to generate content is impressive and will (or rather, already is) being used for practical things.
fwungy 3y ago
The first 80% of AI tasks will be easy enough. The next 20% will be increasingly difficult with the last 1% virtually impossible. Same with driverless cars.
- danielmarkbruce 3y ago
  Folks who are pessimistic on driverless cars appear to move the goals on the last 1% repeatedly. Self driving cars are driving themselves around Phoenix and SF right at this minute.
  - vajrabum 3y ago
    I passed a driverless car from Cruise in San Francisco on my commute just the other day.

jhbadger 3y ago

>AI research is poor. Many of their claims will be proven wrong. This is true of every field of science. As Dara Ó Brian memorably said, "Science knows it doesn't know everything. If it did, it would stop".

MichaelMoser123 3y ago

now is 'AGI is just around the corner' an example of techno-optimism or techno-pessimism?

trashtester 3y ago
Both, and it will positively cause humanity's extinction.
- MichaelMoser123 3y ago
  not sure about that. i suspect that AGI might be a tool to help one of the bigcorps to grab power. Until now the state and the bigcorps were keeping each other in check, there was a kind of 'balance of power'. Now they might get a tool to upset this balance of power. Whoever gets AGI first might turn into Buy-N-Large or ACME Corp, whatever... (i suspect it's all about the big price - ultimate power) - it's kind of the 'ring to rule them all' - if AGI is achievable within a realistic time frame... * now let's see if that's the dystopian future that we are really after :-) *

azubinski 3y ago

"To forestall having a “dasvidaniya comrade” moment where a self-aware GPT-5 shoves an icepick into their ear, Trotsky-style, they put together a ‘red team’ that tested whether GPT-4 was capable of ‘escaping’ or turning on its masters in some way" You can agree with the author, you can disagree with the author, but he has a Style.

hospitalJail 3y ago

The author never used ChatGPT. Its clear from their writing. They would be able to acknowledge the risks, but they pretended it was impossible because the quality isnt perfect. They don't know how to use the tool/never used the tool, but decided to write a book on it? This seems like a trope: "LLMs suck! No I never tried it before"

alostpuppy 3y ago
Alternative pitch… the author used ChatGPT to write this.

agentultra 3y ago

A thousand times this. It's unfortunately too easy to find medieval thinking in tech and computer science circles. Many who are ready and eager to believe rather than remain skeptical. A great number of people who claim the title of researcher or engineer that can only use fractured metaphors and tenuous analogies to extrapolate science fictional predictions from marketing copy. And a great deal of them have plenty of financial incentive to believe and convince others to do so. It was disheartening to see so many in the tech community get taken in by the crypto-craze when all manner of inevitable futures were imagined for us. LLM's are a tool. Data and algorithms. To attribute self awareness and intelligence to them is pure hyperbole.

kalkin 3y ago
Self-awareness and intelligence (in the sense in which AI has any and will likely have more in five years) are not the same thing. My cats have more self awareness than ChatGPT but the latter can do a lot of intellectual tasks that nonhuman animals can't. What's disheartening for me is to see people make confident pronouncements about how other people are being taken in by scifi and hype and there's nothing to worry about even while displaying what seem like basic conceptual confusions.
- agentultra 3y ago
  I'm not worried that GPT-4 is going to "escape its jail, become self aware, and start building a robot army to conquer the world." Or whatever. I wasn't worried that the future of financial markets and payment systems were going to be replaced with crypto-currencies and the IMF replaced with the Ethereum developers. I didn't believe for a second that they would successfully supplant the financial institutions and state governments and escape regulation. We were also supposed to see the modern Internet replaced with web3. It was inevitable! These were all hyperbolic claims made by crypto enthusiasts for years. They were preposterous from day one but it didn't stop many smart, intelligent people from being taken in and converted to believers. There are real dangers! However I think those dangers come from other humans and human institutions. Just as the dangers of crypto came from real humans pulling off huge scams and fraud. Whether it's corporations hyping their AI products to health care professionals and causing thousands of misdiagnoses that lead to chronic health problems or death; to scammers using them to trick family members of their targets to divulge useful information... that's all humans being humans and leveraging a tool that makes it easier for them to cause harm (intentionally or otherwise). What we need are people cutting through the hype so that regular people don't mistake negligence or intentional malice with the "sentience" of "AGI."

xwdv 3y ago

The common man’s perception of AI and its capabilities right now is similar to how people would look at you like you’re the Wizard of Oz just by building a web page back in 1999, and if you were young enough they would even herald you as “the next Bill Gates!”

mountainofdeath 3y ago

"Criti-hype", I like that word. It indeed is like car salesperson hyping that the very thing they are selling is almost not street legal, begging the buyer to try it on and placing that seed of thought. On another note, everyone is now using "algorithms and machine learning" in the same way they used "blockchain" in 2021. The best part is, the majority of the "algorithms" being used are basically glorified Excel models; something you would expect a fresher in college to piece together for their Physics 101 class.

robofanatic 3y ago

These days every software company has AI and ML acronym in their tag lines irrespective of whether or not they have anything to do with it.

jschveibinz 3y ago

We continue to see these “oh it’s not…” and “watch out for…” articles. Let’s take a step back and just take it for what it is. If I walk up to a regular someone on the street and say “please write me an outline for a short story about tuna fish and bunny rabbits,” you’re going to get a blank stare. Heck, even an experienced story teller might have a tough time with that. But these current AI tools will give you a credible outline for that bunny-tuna story in about 10 seconds. It’s not the best story. But it’s a good starting point. To me, that’s freaking great. Maybe it’s a little esoteric, but still amazing. And there are a zillion other things that these tools can produce very quickly that are excellent starting points. Humans can’t do this. Period. There is no snake oil. The tools have instant access to decades of human development. Some of us can’t find Washington, DC on a map let alone write a short story outline or 100 lines of code in 10 seconds. How does that “talent” translate to productivity, i.e. useful output? That’s up to the users. Some people will know how to ask the right questions. Other people won’t. There will definitely be another “sorting” of people based on access and ability to use the tools. But this is the course of human development. It’s evolutionary, really. From now on we go forward with these tools. That’s my take.

sangnoir 3y ago
> If I walk up to a regular someone on the street and say “please write me an outline for a short story about tuna fish and bunny rabbits,” you’re going to get a blank stare. Probably they'd be trying to assess if you're sane and not a danger to them. This is a task a 6 year-old could accomplish with style and vigor.
anotherman554 3y ago
"But these current AI tools will give you a credible outline for that bunny-tuna story in about 10 seconds. It’s not the best story. But it’s a good starting point." A good starting point for what? If the end result is a story that nobody but yourself will ever want to read, why not simply copy and paste "All work and no play makes Johnny a dull boy" a thousand times, no need for an A.I.

greenhearth 3y ago

The main theme of the article for me is the discrepancies between marketing (shiny cable news segments and social media) and real, dirty-hands research and development. They are completely out of sync. I am studying Comp Ling and one of the first understandings you develop is that Natural Language Processing through neural networks can achieve a level of accuracy and efficiency (which is pretty high), but eventual limitations occur where improvement is impossible. And perfection is certainly an impossibility.

helen___keller 3y ago

I never found a novel useful product after many hours of scouring web3 startups. Usually it was “X for web3”, predicated on something else useful existing to justify the web3 ecosystem I’m cautiously more optimistic about AI (this time) since we’ve seen impressive functionality from OpenAI and such, and the availability of an impressive LLM as an API excites possibilities. In my head a large open question is how flexible / powerful fine tuning and context/prompt engineering can be; if it’s not powerful enough, then I predict a wave of snake oil failures, and the space fizzles out for a few more years. On the other hand, if you can take an API, some fine tuning data, and some prompt engineers, and build an actually useful service from that, then the space is about to explode with new unicorns

scp3125 3y ago

This is a tangent, but snake oil was medicinally effective. The colloquialism "snake oil salesman" comes from the old scam of confidence men selling a cheap, adulterated (and sometimes drugged) mineral oil as snake oil (based on the Chinese water snake from Chinese traditional medicine). People would pay a premium for what they thought was medicine, only to find out later that the confidence men bamboozled them and skipped town with their money. https://en.wikipedia.org/wiki/Snake_oil https://en.wikipedia.org/wiki/Snake_oil

iudqnolq 3y ago
From your source: > In 18th-century Europe, especially in the UK, viper oil had been commonly recommended for many afflictions, including the ones for which oil from the rattlesnake (pit viper), a type of viper native to America, was subsequently favored to treat rheumatism and skin diseases.[6] Though there are accounts of oil obtained from the fat of various vipers in the Western world, the claims of its effectiveness as a medicine have never been thoroughly examined, and its efficacy is unknown. (emphasis added)

kriro 3y ago

We might be going full circle soon and in need of "Beware of Beware of AI pseudoscience and snake oil". In my personal bubble there's roughly the same number of AI snake oil stuff popping up as there is finger wiggly "be weary off AI claims" stuff that lacks even the simplest understanding of AI let alone modern things like how transformers actually work.

nottorp 3y ago

Crypto in Rust (on HN and maybe other techie places) AI Anyone want to make predictions on what the next overhyped topic will be in a few years? Maybe we should ask ChatGPT :)

SamPatt 3y ago

I feel like the article would have been a lot more convincing in the pre-GPT era. I use GPT-4 daily, and the author is simply wrong about its capabilities.

gtirloni 3y ago

This showed up in the front page and was quickly downvoted to the second page before I had finished the first page. The Kool Aid is strong today.

daveguy 3y ago
You can't downvote a whole thread, only comments. Threads can only be flagged or upvoted and the thread is not flagged. If there isn't enough interest it fades to the second page after a while. That's what happened here.
- gtirloni 3y ago
  Thanks for the clarification. Does it penalize a submission when someone flags it (but it doesn't reach whatever threshold there is to actually mark it as flagged)? I'm assuming there's a threshold.
  - daveguy 3y ago
    That's a good question and it looks like a flag is considered in ranking before it goes defunct. From the faq [0]: How are stories ranked? ...upvotes [positive]...time [negative]... Other factors affecting rank include user flags, anti-abuse software, software which demotes overheated discussions, account or site weighting, and moderator action. So for all intents and purposes it looks like a flag acts in some way as a downvote. I was just confused by the term "downvote". They don't have the same effect as an upvote (maybe more effect?) but they do drop a story in the rankings even though they may not be removed. Those last three factors look like it can be rather arbitrary. But in general I prefer the quality of articles and discussions here to other sites.

Sloppy 3y ago

Hype is ever-present, beware. OK. ML was overhyped but has proven useful, OK. But the new Generative AIs are fundamentally different. They are trained without any notion of a ground truth. They therefore have no concept of reality. They only know what is real-ish. They are psychotic. Temper the hype with the caveat--"but what it produces is often untruthful or unreal" and the hype is instantly clear. This essay fails to account for why GAI is so different. This problem can only be solved with new methods for training that have as yet not been invented. Older style ML is based on using some ground truth and comes with a metric that measures how close it is to this truth. Not so of any of the new GAI, so continue to beware until researcher figure out how to measure truth--and good luck with that.

RHSman2 3y ago

The pursuit of AI is intellectually interesting however we aren’t intellectually advanced enough to see why it is a very bad idea.

wanderingmind 3y ago

You mean like most so called regular science which is never reproducible as well

kaycey2022 3y ago

This is an article that is both timely and necessary. Way too many people are simply leaning into the AI hype and there is simply not enough scepticism to go with it. I refer to one of the comments made in the thread about the state of self driving cars: “People who haven’t been able to make any progress in self driving cars are now claiming that full conscious AI is just an year away!”