12 ms·

I've trained as a neuroscientist and written a book about consciousness. I've worked in machine learning and built products for over 20 years and now use AI a f

• limit499karma 2y ago
  Why are you throwing in 'consciousness' in a comment regarding mechanical intelligence?
• juanani 2y ago
  [dead]
• phito 2y ago
  > but I am yet to see any reasoned argument for why it is far more difficult and will take far longer I am yet to see any reasoned argument for why it is easy to build real AI and that it will come fast. As you said, AI has been there for decades and stagnated for pretty much the whole time. We've just had a big leap, but nothing says (except BS hype) that we're not in for a long plateau again.
  • bamboozled 2y ago
    Well he gave you a list of credentials of why you should believe him. Isn’t that enough ?
    • phito 2y ago
      No. Stating credentials proves nothing at all. Even less so on the internet. edit: oh sorry I didn't get that it was sarcasm
    • dartos 2y ago
      > I've trained as a neuroscientist and written a book about consciousness This has to do with ML and numerical computing, how?
      • bamboozled 2y ago
        Well I was being sarcastic. I really dislike it when people have to first convince you that you should trust them. Either make a good argument or don’t.
        • lemarchr 2y ago
          Argument from authority is a pernicious fallacy, and typically effective too. You were right to call it out. I must admit I overlooked the sarcasm, however.
          • lelanthran 2y ago
            > I must admit I overlooked the sarcasm, however. Don't feel too bad; until I read the next response I was in two minds about whether sarcasm was intended or not. It's bloody hard to tell, sometimes :-/
            • carlmr 2y ago
              https://en.m.wikipedia.org/wiki/Poe%27s_law https://en.m.wikipedia.org/wiki/Poe%27s_law
• cosmicradiance 2y ago
  On the current state of AI - do you believe it has "intelligence" or is the underlying system a "prediction machine"? What signs do you see that make you believe that the next level (biological intelligence) is on the horizon?
  • zaptrem 2y ago
    We are but prediction machines https://www.psy.ox.ac.uk/news/the-brain-is-a-prediction-machine-it-knows-how-good-we-are-doing-something-before-we-even-try https://www.psy.ox.ac.uk/news/the-brain-is-a-prediction-mach...
    • ChaitanyaSai 2y ago
      We do predictions, but much more important, we are able to create new states. Prediction the classical view assigns probabilities to existing states. What's unique to us and a lot of other biological intelligence is the ability to create new states when needed. This is not implicit in the narrow view of prediction machines
    • bamboozled 2y ago
      Is this a good thing ? Because apparently we’re supposed to be building god. So it sounds like we’re on the wrong track, am I wrong ? If we’ve just copied our feeble abilities, is that supposed to be exciting? Is god like intelligent just a prediction machine too ?
      • giardini 2y ago
        Well, if we intend "building god" perhaps we're merely making a copy of a copy of God: The Sixth Day Then God said, “Let Us make man in Our image, after Our likeness, to rule over the fish of the sea and the birds of the air, over the livestock, and over all the earth itself and every creature that crawls upon it.” So God created man in His own image; in the image of God He created him; male and female He created them. ”… Genesis 1:26-27 Berean Standard Bible
    • cosmicradiance 2y ago
• seoulmetro 2y ago
  What's the next big step? What will it do? Why do we need or want it? Surely you have the answer. This means you are sure we are close to automated driving, engineering and hospitality?
  • edanm 2y ago
    > This means you are sure we are close to automated driving [...] We already have "automated driving" in some sense. Some cities have fully autonomous taxi services that have operated for a year or more, iirc.
    • seoulmetro 2y ago
      Nah. We're still not that close. Think of it this way, you turn on an appliance at home and it's what, a 0.0001% chance it will explode in your face? Now automated driving, hospitality etc is all more like a 0.1+% chance something goes wrong still. Huge difference. I don't really take those taxis as a form of solved automation. It's a nice step though.
      • fragmede 2y ago
        Why don't robotaxis count?
        • seoulmetro 2y ago
          Because large, money-rich companies and censored usage is not a proving ground. Amazon "trialled" stores that were automated but ended up being humans. Even without the human factor they weren't proof of successful automation of retail.
• dartos 2y ago
  > I am yet to see any reasoned argument for why it is far more difficult and will take far longer. For language models specifically, they are trained on data and have historically been improved by increasing the size of the model (by number of parameters) and by the amount and/or quality of training data. We are basically out of new, non-synthetic text to train models on and it’s extremely hard work to come up with novel architecture that performs well against transformers. Those are some simple reasons why it will be far more difficult to improve general language models. There are also papers showing that training models on synthetic data causes “model collapse” and greatly reduces output quality by magnifying errors already present in the model, so it’s not a problem we can easily sidestep. It’s an easy mistake to see something like chatgpt not exist, then suddenly exist and assume a major breakthrough happened, but behind the scenes there has been like 50 years of R&D that led to it, it’s not like suddenly there was a breakthrough and now the gates are open. A general intelligence for CS is like the elixir of life for medicine.
  • slidehero 2y ago
    >We are basically out of new, non-synthetic text to train models this is not even remotely true. There is an astronomical amount of data siloed by publishers, professional journals etc. that is yet to be tapped. OpenAI is making inroads by making deals with these content owners for access to all that juicy data.
    • staticman2 2y ago
      >>>There is an astronomical amount of data siloed by publishers, professional journals etc. that is yet to be tapped. You seem to think these models haven't already been trained on pirated versions of this content, for some reason.
      • dartos 2y ago
        Yep, books3 is what llama was famously trained on before it was taken down. That’s not even considering AI crawlers or all the copyright text on archive.org
    • 2y ago
• whatever1 2y ago
  There is no guarantee that we will not get stuck with these probabilistic parrots for 50 more years. Definitely useful, definitely not AI. And by the way I can copy your post character by character, without hallucinating. So I am definitely better than this crop of "AI" in at least one dimension.
• lasc4r 2y ago
  >We've run the 4-minute mile. >We are in for radical non-linear change. We aren't running miles much quicker than 4 mins though. The last record was 3m:43s set by Hicham El Guerrouj in 1999.
  • blitzar 2y ago
    The 1-minute mile must be right around the corner, and when that inevitably gets broken, the 1-second mile will follow swiftly.
    • XMPPwocky 2y ago
      In fact, humans will be running at relativistic speeds within this century, risking the total destruction of the Earth if they should ever trip over something and fall down. Scary stuff. And it's not science fiction- it's based on real, observed trends and scaling laws. Seems impossible? Well, they said the four-minute mile was impossible, too.
  • PaulRobinson 2y ago
    While this is true, I think you’re not appreciating the metaphor. Humankind tried to break the 4 minute mile for hundreds of years - since measuring distance and time became accurate enough to be sure of both in the mid-18th century, at least - and failed. In May 1954, Roger Bannister managed it. By late June it was done again by a different runner. Within 20 years the record was under 3:45, and today there are some runners who have achieved it more than 100 times and nearly 1800 runners who have done it at all. Impossible for hundred of years, and then somebody did it, and people stopped thinking it was impossible and started doing it themselves. That’s the metaphor: sometimes we think of barriers that are really mental, not real. I’m not sure that applies here either, but the point is not that progress is continuously exponential, but that once a barrier is conquered, we take on a perspective as if the barrier were never real in the first place. Powered flight went through this. Computing hardware too. It’s not an entirely foolish notion.
• randcraw 2y ago
  I've worked in AI for the past 30 years and have seen enthusiasm as robust as yours go bust before. Just because some kinds of narrow AI have done extraordinarily well -- namely those tasks that recognize patterns using connections between FSMs -- does not mean that same mechanisms will continue to scale up to human-level cognition, much less exceed it any time soon. The breakthroughs where deep AI have excelled -- object recognition in images, voice recognition and generation, and text-based info embedding and retrieval -- these require none of the multilevel abstraction that characterizes higher cognition (Kahneman's system 2 thinking). When we see steady progress on such frontiers, only then a plausible case can and should be made that the essentials of AGI are indeed within our grasp. Until then, plateauing at a higher level of pattern matching than we had expected -- which is what we have seen many times before from narrow AI -- this is not sufficient evidence that the other requisite skills needed for AGI are surely just around the corner.
  • spacemanspiff01 2y ago
    So I am a neophite in this area, but my thesis for why "this time is different" compared to previous AI bubbles is that this time there exist a bunch of clear products (or paths to products) that work and only require what is currently available in terms of technology. Coding assistants today are useful, image generation is useful, speach recognition/generation is useful. All of these can support businesses, even in their current (early) state. Those businesses have value in funding even 1% improvements in engineering/science. I think that this is different than before, where even in the 80s there were less defined products, amd most everything was a prototype that needed just a bit more research to be commercially viable. Where as in the past, hopes for the technology waned and funding for research dropped off a cliff, today's stuff is useful now, and so companies will continue to spend some amount on the research side.
    • bamboozled 2y ago
      I don't find coding assistants to be very useful. Image generation was fun for a few weeks. Speech recognition is useful. Anyway, considering all these things can be done on device, where is the long term business prospect of which you speak?
• seydor 2y ago
  From a neuroscience perspective , current AI has not helped explain much about real brains. It did however validate the connectionist model of intelligence and memory, to the point that alternate theories are much less believable nowadays. It is interesting to watch the deep learning field evolve, hoping that at some point it will intersect with brain anatomy.
• 0points 2y ago
  > There are hundreds of billions of dollars figuring out how to get to the next level, and it's clear we are close. Are we really now? The smart people I've spoken to on the subject seem to agree the current technology based on LLM are at the end of the road and that there are no breakthrough in sight. So what is your take on the next level?
  • KoolKat23 2y ago
    Define breakthrough, there's plenty of room to scale and optimize without any need for a breakthrough (well my definition of breakthrough). Emergent properties so far have been obtained purely from scaling.
    • slashdave 2y ago
      There will be no more progress via scaling. All the available training data has already been exploited.
• floppiplopp 2y ago
  When Weizenbaum demonstrated Eliza to his colleagues, some thought there was an intelligent consciousness at the heart of it. Few even continued to believe this after they were shown the source code, which they were able to read and understand. Human consciousness is full of biases and the most advanced AI cannot reliably determine which of two floats is bigger or even solve really simple logic puzzles for little kids. But I can see how these things mesmerize true believers.
  • hnfong 2y ago
    At this point bringing up the ELIZA argument is basically bad faith gaslighting… Finding bugs in some models doesn’t mean you have a point about intelligence. If somebody could apply a similar argument to dismiss human intelligence, you don’t have a point. And here it goes: the most advanced human intelligence can’t reliably multiply large numbers or recall digits of Pi. Obviously humans are dumber than pocket calculators.
    • psb217 2y ago
      Your counterargument is invalid. The most advanced human intelligence invented (or discovered) concepts like multiplication, pi, etc., and created tools to work around the ways in which these concepts aren't well handled by their biological substrate. When machine intelligences start inventing biological tools to overcome the limits of their silicon existence, you'll have a point.
      • biomcgary 2y ago
        Isn't the comment you are responding to an example of: "When machine intelligences start inventing biological tools to overcome the limits of their silicon existence, you'll have a point"?
      • mypalmike 2y ago
        Designing biological tools is not a commonly accepted bar for AGI.
  • 2y ago
• benterix 2y ago
  > and it's clear we are close I'd like to believe it more than you do. Unfortunately, in spite of these millions of dollars, the progress on LLMs has stalled.
• pdimitar 2y ago
  > And I must say I am absolutely astonished people don't see this as opening the flood-gates to staggeringly powerful artificial intelligence. This looks like a cognitive dissonance and they are addressed by revisiting your assumptions. No flood-gates have been opened. ChatGPT definitely found uses in a few areas but the number is very far from what many people claimed. A few things are really good and people are using them successfully. ...But that's it. Absolutely nothing even resembling the beginnings of AGI is on the horizon and your assumption that the rate of progress will remain the same -- or even accelerate -- is a very classic mistake of the people who are enthusiasts in their fields. > There are hundreds of billions of dollars figuring out how to get to the next level, and it's clear we are close. This is not clear at all. If you know something that nobody else does, please let us know as well.
• reportgunner 2y ago
  > I am absolutely astonished people don't see this as opening the flood-gates to staggeringly powerful artificial intelligence. Perhaps it's confirmation bias ?
• grantcas 2y ago
  [dead]
• surfingdino 2y ago
  What are you talking about? What autonomy? Try the latest Gemini Pro 1.5 and ask it for the list of ten places to visit in Spain. Then ask it for the Google Maps URLs for those places. It will make up URLs that point to nowhere. This os of zero value for personal or business use. I have dozens of examples of such crappy outcomes from all "latest", "most powerful" products. AI is smoke and mirrors. It is being sold as a very expensive solution to a non-existent problem and is not getting any better in the future. Some wish AI had someone like Steve Jobs to properly market it, but even Steve Jobs could not make a crappy product sell. The whole premise of AI goes against what generations of users were told--computers always give correct answers and given the same input parameters return the same output. By extension, we were also taught that GIGO (Garbage-In, Garbage-Out) is what we can blame when we are not happy with the results computers generate. AI peddlers want us to believe in and pay for VIGO (Value-In, Garbage-Out) and I'm sorry but there is not a valid business model where such tools are required.
• matteoraso 2y ago
  I don't know anything about neuroscience, but is there anything in the brain even remotely like the transformer architecture? It can do a lot of things, but I don't think that it's capable of emulating human intelligence.
  • warkdarrior 2y ago
    I don't know anything about biology, but is there anything in birds even remotely like the airplane? It can do a lot of things, but I don't think that it's capable of emulating bird flight.
• greenthrow 2y ago
  You sound like you don't actually understand anything about LLMs and are buying into the hype. They are not cognizant let alone conscious. They don't understand anything. The tokens could be patterns of colored shapes with no actual meaning, only statistical distributions and nothing about how the LLMs work would change.
  • stevenhuang 2y ago
    > The tokens could be patterns of colored shapes with no actual meaning, only statistical distributions and nothing about how the LLMs work would change. I can put your brain in a vat and stimulate your sensory neurons with a statistical distribution with no actual meaning, and nothing about how your brain works would change either. The LLM and your brain would attempt to interpret meaning with referent from training, and both would be confused at the information-free stimuli. Because during "training" in both cases, the stimuli received from the environment is structured and meaningful. So what's your point? By the way, pretty sure a neuroscientist with 20 years of ML experience has a deeper understanding of what "meaning" is than you do. Not to mention, your response reveals a significant ignorance of unresolved philosophical problems (hard problem of consciousness, what even is meaning) which you then use to incorrectly assume a foregone conclusion that whatever consciousness/meaning/reasoning is, LLMs must not have it. I'm partial to doubting LLMs as they are now have the magic sauce, but it's more that we don't actually know enough to say otherwise, so why state that we do know? We can't even say we know our own brains.
    • greenthrow 2y ago
      Your response is nonsense. We don't know how consciousness arises from matter but we do have significant understandings about knowledge, reasoning, modeling, visualization, object permanence, etc. etc. that are significant parts of how the human mind works. And we know LLMs have none of these. The point of my colored shape example is that it is an illusion that there is anything resembling a mind inside an LLM. I thought that was obvious enough I didn't need to explain it further than I did. As far as the original commenter's credentials; there's lots of people who should know better but buy into hype and nonsense.
• layer8 2y ago
  We don’t know what the next big leap will bring and when it will happen. The occurrence of a singular previous big leap cannot serve as any reliable predictor.
• washadjeffmad 2y ago
  Agreed. This is something I didn't think I'd see in my lifetime, let alone be poised to be able to run locally. The alignment is fortuitous and staggering. People focused on the products are missing out on the dawn of an epoch. It's a failure of perspective and creativity that's thankfully not universal.
• ypeterholmes 2y ago
  Awesome comment. I've written a piece here about the relationship between AI and human consciousness. Would love some feedback if you're able. Thanks! https://peterholmes.medium.com/the-conscious-computer-af5037439175 https://peterholmes.medium.com/the-conscious-computer-af5037... PS. I'm buying your book right now.
• sandspar 2y ago
  The potential rewards are so great that you might be overestimating the odds this will come about. Even lottery skeptics might buy a lottery ticket if the prize is a billion dollars.
• nprateem 2y ago
  It's obvious there's potential. It's also obvious it requires at least one other major breakthrough. But no one knows how far away that is.
  • deleted 2y ago
    [deleted]
• dogcomplex 2y ago
  Fascinating background - would love to pick your brain on how you see current LLMs/ML comparing to neuroscience. What do you see that's missing still, if anything? If I had to bet, I would start with: - Error-correcting specialized architectures for increasing signal-to-noise (as far as I can tell these are what everyone is racing to build this year, and should be doable with just conventional programming systems wrapping LLMs) - Improved energy efficiency (as yes, human brains are currently much more efficient! But - there are also simple architecture improvements (both software and hardware) that are looking to save 100x. Specialized ASIC ternary chips using 1999's tech should be here quite soon, a lot more efficient in price and energy.) - No Backwards-propagation. (As yes, the brain does seem to do it all with forward-propagation only. Though this is possible and promising in neural networks like the Forward-Forward algorithm too, they haven't been trained to the same scales as backprop-heavy transformers (and likely have a lower performance in terms of noise/accuracy). Though if I'm not mistaken, the brain does have forward-backward loops, but the signals go through separate neurons for each direction (rather than reusing one) - if so that's close to backprop by itself, but probably imposes a tradeoff as the same signal can't be perfectly reproduced backwards, yet it can perhaps be enhanced to be just the most relevant information by the separate specialized neuron. I'm obviously mostly ignorant of the neuroscience here but halfway-knowledgeable on the ML theory haha But yes, I completely agree - the flood gates are already open. This is a few architecture quibbles away from an absolute deluge of artificial intelligence that will dwarf (drown?) anything we've known. Good point on decentralized cheap autonomy - the real accomplishment of life. Intelligence, as it appears, is just a fairly generous phenomenon where any autonomous process continually improves its signal-to-noise ratio... many ways to accomplish that one! Looking forward to seeing LLMs powered by ant colonies and slime molds, though I suspect by then there will be far more interesting and terrifying realities unlocked.
• fire_lake 2y ago
  The argument that we are going to see massive progress soon is weak in my view. It seems to be: - we had some big breakthroughs recently - some AI “godfathers” are “really worried”
• IAmGraydon 2y ago
  >I've trained as a neuroscientist Can you explain what this means? Do you have a degree in neuroscience?
• Breza 2y ago
  > Just like with powered-flight we don't need bioliogical intelligence to transform the world. Powered flight offers a cautionary tale for AI. The first confirmed powered flight was in 1903. For the next 60 years, someone broke the airspeed record almost every year. The current record was set in 1976. Nobody has broken that record for 48 years. There are concerns that the state of AI will show a similar pattern, with rapid improvements followed by a plateau.