10 ms·

The Lone Banana Problem in AI

146 points by JohnHammersley 3y ago

auggierose 3y ago
> A single banana casting a shadow on a grey background That prompt works fine in DALL·E 2.
- compiler-guy 3y ago
  It will likely be fixed in the next iteration of MidJourney as well. But the point isn’t about specific prompts where the error is obvious and relatively easy to fix. This is just a n extremely easy example to understand. The point is about the subtle biases that are much harder to detect, and which will therefore go unfixed.
  - cubefox 3y ago
    Text-to-image models have at least two dimensions of performance: Image quality and text understanding. Midjourney is outstanding at image quality, since it is able to make pictures with few "artifacts", like messed up hands. But it isn't so great at understanding text. For example, the new Dall-E version used by Bing is significantly better than Midjourney in producing an "invisible monkey", although far from perfect: https://www.bing.com/images/create/an-invisible-monkey-with-a-single-banana/64a3e3fac82d48a7af136b64223aeac3 https://www.bing.com/images/create/an-invisible-monkey-with-... (Adobe Firefly also fails the invisible monkey task, despite usually having better image quality than Bing/Dall-E.)
- rob74 3y ago
  I can understand the logic behind why "a single banana" doesn't prevent two bananas from showing up though: there is a single banana casting a shadow on a grey background in the image. And then there is another one. That's why you have to specify "on its own" to get rid of the second one...
  - auggierose 3y ago
    If you look at the examples in the post, the two bananas are usually joined, so definitely not single. So I think your explanation is overthinking it.
kthejoker2 3y ago
Seems like we need more actor crific models that evaluate theae outputs on real world physical modeling and accuracy and not just quality of artistic output or token similarity. But also Midjourney in particular seems trained to be more evocative / stylistic rather than photorealistic or precise.
generationP 3y ago
Maybe he should try "bananum"?
tudorw 3y ago
I've seen lots of long prompt suggestions, however single words are fascinating, some words are extremely heavy, for example, we might assume 'topic' and 'subject' are somewhat interchangeable, but when I'm talking to an LLM swapping those words cause a huge change. Same with names, if you ask it to act like Dave, it's not some random personality, it's a distillation of every Dave it has encountered! I don't know how we could visualise and explore this but it's certainly interesting.
- sebzim4500 3y ago
  I wonder how much of this is due to mode collapse. With RLHF, there isn't really anything stopping it deciding that all Daves have a certain personality.
- kgwgk 3y ago
  These are the Daves I know: https://m.youtube.com/watch?v=8nvzEqsZIGo https://m.youtube.com/watch?v=8nvzEqsZIGo
  - bluenose69 3y ago
    Somehow, I needed that blast from the past!
- isaacfung 3y ago
  On /r/midjourney subreddit there are posts like "the most stereotypical person in [state/country]", "what midjourney thinks professors look like based on their department". You can see some interesting biases in the training data. https://www.reddit.com/r/midjourney/top/?sort=top&t=year https://www.reddit.com/r/midjourney/top/?sort=top&t=year
  - cubefox 3y ago
    I wouldn't use "bias" here. It's a very ambiguous term in machine learning.
    - 3y ago
mrchumphatty 3y ago
Blog post by company set up by corporate publisher solely for data mining purposes = shill bananas.
rishabhjain1198 3y ago
I wish this article was just 3 paragraphs. The verbose writing style was a little tiring, I found myself scrolling impatiently to find what the actual "Lone Banana Problem" was.
- oniony 3y ago
  I skipped to Going Bananas, speed read the rest and it was fine.
- deleted 3y ago
  [deleted]
- denton-scratch 3y ago
  Agree. I was a couple of pages in before I learned what the "Lone Banana Problem" is. That took two paras, and was followed by a lot of sophomoric philosophical noodling.
  - DemeterFarm 3y ago
    [dead]
- bunga-bunga 3y ago
  Someone needs to create a browser that parses and displays every website in a format that I request. My preferred format would probably be: - answer to the title / synopsis - the article, without ads, using the site’s style
- lproven 3y ago
  The pictures did not immediately tell you? They told me what the problem was in about 2 seconds, and frankly, the title and the first sentence alone were. All right, I concede, a blind reader would not know, but most people are not blind.
  - foobarbecue 3y ago
    I read almost the whole article thinking there might be something more in there. There was mention of TCAV but that was about it.
thriftwy 3y ago
Humans are in the business of consuming bananas, whereas neural nets are in the business of peddling bananas. They don't get to actually use these bananas so they can't gain deeper insight into what's they for. This is the classic lamb vs. mutton issue. Wealthy land owners who use one set of idioms vs. servants who use a different one. Happens to neural nets on human chassis as well.
- pavlov 3y ago
  > “Humans are in the business of consuming bananas” Someone should have told Jean-Paul Sartre this. He didn’t have to be miserable wondering what it’s all about.
- brazzy 3y ago
  That touches on something I've pondered for a while: Maybe the popular sci-fi image of "artificial brains" with humanoid bodies actually turns out to be more accurate than the technical view that robotics and artificial intelligence are mostly separate fields! What if, instead of machine learning being a source of features we can maybe later build into robots, it's the other way round? What if direct interaction with the real world through robotic limbs and sensors turns out to be what it takes to get AI to stop hallucinating? What if, furthermore, having a body turns out to be an essential requirement for AI to develop a sense of self? Not just because of sensory feedback, but simply because it provides an anchor point for identity?
  - iHeartVLC 3y ago
    [dead]
codegladiator 3y ago
Filling in the blank with what has not happened. LLMs interpolate quite well but they don't extrapolate. Very interesting article where you can or cannot put LLMs to use. It's not necessarily a bug, but we are yet to see.
- goldenkey 3y ago
  They extrapolate just fine. It's the image nets that don't. Here's my excerpt with GPT-4: As an AI, I can't actually see the painting, but based on the title and the dimensions you've provided, I can imagine a possible description. The painting, titled "Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style," is a large piece, standing 6 feet tall and 4 feet wide. At the bottom of the painting, the trenchcoat's hem brushes the ground, obscuring the cats' feet. The trenchcoat is a deep, rich color, perhaps a classic trenchcoat beige or a more dramatic black or navy. The coat is painted with careful detail, showing the texture of the fabric and the gleam of buttons. Moving up the painting, the middle section reveals the outline of three cats stacked on top of each other within the coat. The artist might have chosen to show this through a slightly open coat, or perhaps through the distinctive shapes visible under the fabric. The cats could be of different breeds and colors to add visual interest and humor. The top third of the painting, the upper two feet, features the head of the topmost cat peeking out from the collar of the trenchcoat. The cat might be looking straight ahead, trying to maintain its human disguise, or it might be looking out at the viewer with a humorous or mischievous expression. If the artist chose to include a hat, it would be perched on the cat's head, adding to the disguise. The background of the painting could vary greatly depending on the artist's style and intentions. It could be a simple, solid color to keep the focus on the cats, or it could be a detailed scene that adds to the story - perhaps a city street to tie in with the Vincent Adultman reference, or a domestic setting to contrast with the cats' unusual behavior. Again, this is a hypothetical description and the actual painting could be very different.
  - Der_Einzige 3y ago
    Most of what you think is extrapolation is interpolation. Models do NOT extrapolate unless they are given access to resources/data that is not somehow in their training dataset. We have done so much encoding of knowledge that a majority of serious use-cases for LLMs are Interpolation, but the fact that we even need retrieval augmented generation shows that they do not extrapolate.
llogiq 3y ago
Again, there is a lot of words to describe the fact that machine learning is just lossy compression for a bunch of data with the possibility to interpolate between data points and get somewhat plausible results. This means data points may get lost during compression/training, and certain things will look off, whether it be a preference for banana pairs, even numbers of fingers or certain weasel words in verbiage.
- bippihippi1 3y ago
  all gradient learning creates kernel machines
- goldenkey 3y ago
  And is RL not? All of these models are constrained by finite weights and then tuned. Are you suggesting we grow a neural network until certain criterion are met with regard to out of distribution test criteria? Hmm
  - llogiq 3y ago
    What would that buy you? You train until you push your loss under a certain threshold, then check the external criteria and if they don't hold you train again? Your external criteria would essentially become another part of your loss function, but the whole training would become vastly more inefficient. I'm saying that we shouldn't expect the models to come up with things we didn't train them to come up with.
    - goldenkey 3y ago
      Yeah, it wouldn't buy us anything additional. The only way to really verify would be to look at out of distribution inputs and see if they satisfy what we'd really like. Wouldn't that be true generalization? Otherwise our model architectures or training data is insufficient.
anotheryou 3y ago
Midjourney solution: "--no bunch, two, multiple": https://i.imgur.com/QnJmLr0.jpeg https://i.imgur.com/QnJmLr0.jpeg These models have a tendency to move towards the average, especially if unprompted. As we see here, sometimes even if prompted otherwise. They just have to get better or need a more precise interface like the —no :). We also couldn't have "a man crawling" before and now we can: https://i.imgur.com/ycVpk3i.jpeg https://i.imgur.com/ycVpk3i.jpeg
- RugnirViking 3y ago
  --no is too strong, it ruins the quality because of negative weight pollution. Something like :: bunch, two, multiple::-0.3 would probably be enough without making them look worse
- jonplackett 3y ago
  They don't really look like bananas anymore though...
  - anotheryou 3y ago
    first one is fine. And I could do it more carefully :) Adjusted and with help of the other commenter°: https://i.imgur.com/e2aE1bQ.jpeg https://i.imgur.com/e2aE1bQ.jpeg And you always have some crap generations :) ° https://news.ycombinator.com/item?id=36584831 https://news.ycombinator.com/item?id=36584831
- dafoex 3y ago
  I agree. While the article is correct that this problem of subtle bias exists, the solution for most cases is to realise that the computer "does what you say, not what you mean".
tsukikage 3y ago
"Three cats in a trenchcoat" is a good one. Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style. You can get them standing side by side wearing trenchcoats. You can get a cat pyramid or a cat totem or a stack or tower of cats (though often the ability to count to three is then lost). You can't get them to share the trenchcoat. Nothing like that occurred in any training set, and the AIs do not understand spatial relationships between objects ("X is on top of Y, inside Z") so you cannot describe how to arrange the things it does know about in the scene. Dall-E won't do it. Midjourney won't do it. Stable Diffusion won't do it. Eventually, enough images will be seeded into the training sets for this to stop being a useful test. But right now, it gives one a fascinating window on what happens when you try to extrapolate outside the cloud of thingspace described by the training data, rather than just interpolating within it. https://www.toothycat.net/~sham/trenchcoats https://www.toothycat.net/~sham/trenchcoats
- goldenkey 3y ago
  It's mostly the generation engine, not the logical AI. Here's what GPT-4 says when asked to describe. > As an AI, I can't actually see the painting, but based on the title and the dimensions you've provided, I can imagine a possible description. > The painting, titled "Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style," is a large piece, standing 6 feet tall and 4 feet wide. > At the bottom of the painting, the trenchcoat's hem brushes the ground, obscuring the cats' feet. The trenchcoat is a deep, rich color, perhaps a classic trenchcoat beige or a more dramatic black or navy. The coat is painted with careful detail, showing the texture of the fabric and the gleam of buttons. > Moving up the painting, the middle section reveals the outline of three cats stacked on top of each other within the coat. The artist might have chosen to show this through a slightly open coat, or perhaps through the distinctive shapes visible under the fabric. The cats could be of different breeds and colors to add visual interest and humor. > The top third of the painting, the upper two feet, features the head of the topmost cat peeking out from the collar of the trenchcoat. The cat might be looking straight ahead, trying to maintain its human disguise, or it might be looking out at the viewer with a humorous or mischievous expression. If the artist chose to include a hat, it would be perched on the cat's head, adding to the disguise. > The background of the painting could vary greatly depending on the artist's style and intentions. It could be a simple, solid color to keep the focus on the cats, or it could be a detailed scene that adds to the story - perhaps a city street to tie in with the Vincent Adultman reference, or a domestic setting to contrast with the cats' unusual behavior. > Again, this is a hypothetical description and the actual painting could be very different.
lionkor 3y ago
> Then think of all the wealth disparity that has been introduced into our world. Think of the social anxiety of always being online. Think of the undermining of our democratic institutions. How is this at all whatsoever needed
- Joker_vD 3y ago
  Well, it's a 2023 piece about AI so of course a paragraph or two of sociological blabbing is required.
- Nasrudith 3y ago
  It isn't really. But those are the hivemind memes and not agreeing with their bullcrap gets you looked at like you are a crazy person or worse, not part of the in crowd. High school has done some serious and lasting damage to socialization throughout the world as social backbiting popularity bullshit became the norm. Not to mention complaining about wealth disparity from investment as if it is a sin of the technology is asinine. If money can produce better results you will see wealth disparity from the simple feedback loop of "more money -> better results -> more money". Your neighbor taking up jogging while you sit on the couch all day is increasing a health disparity, should your neighbor stop jogging then? Undermining of democratic institutions is just plain a dumb meme from vested interests who cast what were previously abuses or necessary evils of the system at best as integral to democracy. We would all recognize blaming the media for undermining the Catholic Church by documenting their child sexual abuse as batshit insane. Yet somehow it is different when it comes to newspapers moaning about their loss of monopolies and influence of cliques.
avidphantasm 3y ago
[flagged]
frankreyes 3y ago
LLMs don't understand meaning, they're just a statistical random number generator.
- dr_hooo 3y ago
  You're just a statistical number generator...
RugnirViking 3y ago
If you want an imo more interesting example try "bulldozer". It's missing a quite... Important piece to our conception of those machines
lordnacho 3y ago
LLM doesn't know stuff, it has few models beyond an extremely deep association between (in this case) descriptions and pictures. Humans have shitty models for stuff, but we have them. We lack the massively deep numerical associations. It can't be that long before someone makes an AI that says knows a few common models like "stuff made of steel is rigid, stuff made of cloth is soft, etc". Along with "If someone keeps emphasizing a number, that's the number of things he wants".
- im3w1l 3y ago
  Me: A designer has created two artpieces. The first one "The gattoya" is made of steel bars welded into a spiral shape. The second "Misiblur" is made of cloth and somewhat resembles a butterfly. Which is most comfortable to rest your head on and why? Chatgpt: Based on the description provided, it is likely that "Misiblur," the art piece made of cloth resembling a butterfly, would be more comfortable to rest your head on compared to "The gattoya," which is made of steel bars welded into a spiral shape. The reason for this is that cloth generally has a softer and more cushioned surface compared to steel bars. When resting your head on Misiblur, the fabric would provide a certain level of support and padding, which could make it more comfortable for resting your head. On the other hand, resting your head on steel bars, as in the case of The gattoya, would likely be less comfortable due to the hardness and lack of cushioning provided by the material. Steel bars are not designed to be soft or ergonomic for resting purposes, so it may not provide the desired comfort. It's important to note that the comfort level can vary depending on individual preferences and the specific design and construction of the art pieces. However, based on the given materials and descriptions, Misiblur made of cloth would generally be the more comfortable option for resting your head.
  - lordnacho 3y ago
    Yes but that's text, and there's plenty of text that will tell you that steel is rigid, and rigid is uncomfortable. Also you gave it an A/B choice, random chance is 50%. Does it insist it's right if you prompt it the other way? Does it make pictures of people who are comfortable in one but not the other? Most of the time when I see something funny in LLMs it's in the images. People with 8 fingers, elbows bent backwards, that kind of thing. Spatial models.
sarabande 3y ago
Here is a summary, part ChatGPT and part me: The “Lone Banana Problem” describes subtle biases of Large Language Models (LLMs) in AI: LLMs reproduce the statistical average of the inputs that they have consumed in the context of the question they have been asked. It's called that problem because the model used to generate images has never seen an individual banana, so when prompted always generates two bananas.
bugmen0t 3y ago
Would be kinda funny if the lone bananas have been removed from the training dataset because they are not part of the depicted item but are merely in the picture for scale :)
maltelandwehr 3y ago
„A single banana —-v 5.2“ in Midjourney gives me flur pictures. Two of these pictures only contain a single banana.
__s 3y ago
They didn't try asking for half a banana?
IIAOPSW 3y ago
or it was always just a dice roll, and asking the same prompts again will give you sometimes 1, usually 2, sometimes 3 bananas.
FrustratedMonky 3y ago
Whatever the critiques of length, this is a good write up on 'bias', that is not about race or politics, and thus a good neutral example about how training sets can skew results in un-intended ways. I didn't think it was that long, it might be that some are cherry picking the points they find interesting and think the rest could have been edited out. I could have stood even more discussion on how this 'blind spot' in the AI model is very much akin to our own blind spots.
GuB-42 3y ago
I remember about 10 years ago in the context of using neural network for image classification, they had a study of what the AI actually saw that justified its decision. This, by the way, is how we got "Deep Dream". For "dumbbell", for the AI, no dumbbell was complete without a muscular arm holding it. That's because in most images in the training dataset have the dumbbells being held, so the system integrated the arm in the pattern. I guess it is the same idea here, most pictures of bananas show several of them, so for the AI, bananas are things that don't go alone.
eimrine 3y ago
I used to experience a very similar problem [1]. They say NNs are bad at counting. But at least your picture has kind of a single piece of bananas, my problem remained unsolved. https://news.ycombinator.com/item?id=32875215 https://news.ycombinator.com/item?id=32875215
theknocker 3y ago
[dead]
crosen99 3y ago
> AIs, at their current level of development, don’t perceive objects in the way that we do – they understand commonly occurring patterns. You see this claim everywhere - that AI operates on statistics and patterns and not actual understanding. But human understanding is entirely about statistics and patterns. When a human sees a collection of particles and recognizes it as, say, a car, all they are doing is recognizing the car-like patterns in how the particles are organized that have a strong statistical correlation with prior observations of things classified as a car. Am I missing something?
- lamontcg 3y ago
  We're not (initially anyway) trained on photography and literature and reddit, our first experience of a banana is probably eating one.
- lolc 3y ago
  After a decade of billions invested in systems with models trained for object recognition in traffic, these models still struggle with object permanence. Why should we expect some rando painter models to do better? They'll paint a lamppost through a car and not notice anything wrong with it. The day where these models can show shoes under a fence and a reflection of the person behind the fence in an opposing shop window? That day will come. But not on the current crop.
BryanLegend 3y ago
I unsubscribed from Midjourney because I couldn't direct it to place an object in a specific place. It's impossible to do something like, "A sphere on the left of the picture." and have it understand it. After having iterative conversations in ChatGPT, Midjourney felt like it has an extremely poor grasp of language.
chaosjevil 3y ago
TL;DR: AI doesn't understand human language prompts, it's just associating specific tokens with specific outputs.
13years 3y ago
At Digital Science, we believe that we have a responsibility to ensure that the technologies that we release are well tested and well understood. The use cases where we deploy AI have to be appropriate for the level at which we know the AI can perform and any functionality needs to come with a “health warning” so that people know what they need to look for – when they can trust an AI and when they shouldn’t. We don't even understand ourselves and we hope to model AI alignment in some image of humanity with the goal that it will be just as benevolent as our fractured war eager society. Yes it is a paradox indeed. I submit there is likely a limitation to how much theoretically AI could ever improve beyond ourselves in the regards to bias. I've described this as the AI Bias Paradox - https://www.mindprison.cc/p/ai-the-bias-paradox https://www.mindprison.cc/p/ai-the-bias-paradox
deleted 3y ago
[deleted]
Der_Einzige 3y ago
This isn't a real problem anymore. Composable Diffusion, Regional Prompting, and Controlnet literally solved all of these issues. The author shows what the world is learning, "Johnny can't prompt", and those who learn how to use Generative AI well are going to be continuing to propel up in their careers while the general public incorrectly concludes that the tools "don't work".
gweinberg 3y ago
It's really weird that the monkey is eating two bananas at once, I would think in training data monkeys would almost always be eating one banana at a time. I've been told that in the real world monkeys and chimps don't peel bananas when they eat them, they eat the peel and all, but i don't know if it's true. Whatever happens in the real world, drawings always show monkeys peeling the bananas as they eat them and I would expect a prompt "show a monkey eating a banana" to show it eating a single peeled banana.

> A single banana casting a shadow on a grey background That prompt works fine in DALL·E 2.

compiler-guy 3y ago
It will likely be fixed in the next iteration of MidJourney as well. But the point isn’t about specific prompts where the error is obvious and relatively easy to fix. This is just a n extremely easy example to understand. The point is about the subtle biases that are much harder to detect, and which will therefore go unfixed.
- cubefox 3y ago
  Text-to-image models have at least two dimensions of performance: Image quality and text understanding. Midjourney is outstanding at image quality, since it is able to make pictures with few "artifacts", like messed up hands. But it isn't so great at understanding text. For example, the new Dall-E version used by Bing is significantly better than Midjourney in producing an "invisible monkey", although far from perfect: https://www.bing.com/images/create/an-invisible-monkey-with-a-single-banana/64a3e3fac82d48a7af136b64223aeac3 https://www.bing.com/images/create/an-invisible-monkey-with-... (Adobe Firefly also fails the invisible monkey task, despite usually having better image quality than Bing/Dall-E.)
rob74 3y ago
I can understand the logic behind why "a single banana" doesn't prevent two bananas from showing up though: there is a single banana casting a shadow on a grey background in the image. And then there is another one. That's why you have to specify "on its own" to get rid of the second one...
- auggierose 3y ago
  If you look at the examples in the post, the two bananas are usually joined, so definitely not single. So I think your explanation is overthinking it.

Seems like we need more actor crific models that evaluate theae outputs on real world physical modeling and accuracy and not just quality of artistic output or token similarity. But also Midjourney in particular seems trained to be more evocative / stylistic rather than photorealistic or precise.

generationP 3y ago

Maybe he should try "bananum"?

tudorw 3y ago

I've seen lots of long prompt suggestions, however single words are fascinating, some words are extremely heavy, for example, we might assume 'topic' and 'subject' are somewhat interchangeable, but when I'm talking to an LLM swapping those words cause a huge change. Same with names, if you ask it to act like Dave, it's not some random personality, it's a distillation of every Dave it has encountered! I don't know how we could visualise and explore this but it's certainly interesting.

sebzim4500 3y ago
I wonder how much of this is due to mode collapse. With RLHF, there isn't really anything stopping it deciding that all Daves have a certain personality.
kgwgk 3y ago
These are the Daves I know: https://m.youtube.com/watch?v=8nvzEqsZIGo https://m.youtube.com/watch?v=8nvzEqsZIGo
- bluenose69 3y ago
  Somehow, I needed that blast from the past!
isaacfung 3y ago
On /r/midjourney subreddit there are posts like "the most stereotypical person in [state/country]", "what midjourney thinks professors look like based on their department". You can see some interesting biases in the training data. https://www.reddit.com/r/midjourney/top/?sort=top&t=year https://www.reddit.com/r/midjourney/top/?sort=top&t=year
- cubefox 3y ago
  I wouldn't use "bias" here. It's a very ambiguous term in machine learning.
  - 3y ago

mrchumphatty 3y ago

Blog post by company set up by corporate publisher solely for data mining purposes = shill bananas.

rishabhjain1198 3y ago

I wish this article was just 3 paragraphs. The verbose writing style was a little tiring, I found myself scrolling impatiently to find what the actual "Lone Banana Problem" was.

oniony 3y ago
I skipped to Going Bananas, speed read the rest and it was fine.
deleted 3y ago
[deleted]
denton-scratch 3y ago
Agree. I was a couple of pages in before I learned what the "Lone Banana Problem" is. That took two paras, and was followed by a lot of sophomoric philosophical noodling.
- DemeterFarm 3y ago
  [dead]
bunga-bunga 3y ago
Someone needs to create a browser that parses and displays every website in a format that I request. My preferred format would probably be: - answer to the title / synopsis - the article, without ads, using the site’s style
lproven 3y ago
The pictures did not immediately tell you? They told me what the problem was in about 2 seconds, and frankly, the title and the first sentence alone were. All right, I concede, a blind reader would not know, but most people are not blind.
- foobarbecue 3y ago
  I read almost the whole article thinking there might be something more in there. There was mention of TCAV but that was about it.

thriftwy 3y ago

Humans are in the business of consuming bananas, whereas neural nets are in the business of peddling bananas. They don't get to actually use these bananas so they can't gain deeper insight into what's they for. This is the classic lamb vs. mutton issue. Wealthy land owners who use one set of idioms vs. servants who use a different one. Happens to neural nets on human chassis as well.

pavlov 3y ago
> “Humans are in the business of consuming bananas” Someone should have told Jean-Paul Sartre this. He didn’t have to be miserable wondering what it’s all about.
brazzy 3y ago
That touches on something I've pondered for a while: Maybe the popular sci-fi image of "artificial brains" with humanoid bodies actually turns out to be more accurate than the technical view that robotics and artificial intelligence are mostly separate fields! What if, instead of machine learning being a source of features we can maybe later build into robots, it's the other way round? What if direct interaction with the real world through robotic limbs and sensors turns out to be what it takes to get AI to stop hallucinating? What if, furthermore, having a body turns out to be an essential requirement for AI to develop a sense of self? Not just because of sensory feedback, but simply because it provides an anchor point for identity?
- iHeartVLC 3y ago
  [dead]

codegladiator 3y ago

Filling in the blank with what has not happened. LLMs interpolate quite well but they don't extrapolate. Very interesting article where you can or cannot put LLMs to use. It's not necessarily a bug, but we are yet to see.

goldenkey 3y ago
They extrapolate just fine. It's the image nets that don't. Here's my excerpt with GPT-4: As an AI, I can't actually see the painting, but based on the title and the dimensions you've provided, I can imagine a possible description. The painting, titled "Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style," is a large piece, standing 6 feet tall and 4 feet wide. At the bottom of the painting, the trenchcoat's hem brushes the ground, obscuring the cats' feet. The trenchcoat is a deep, rich color, perhaps a classic trenchcoat beige or a more dramatic black or navy. The coat is painted with careful detail, showing the texture of the fabric and the gleam of buttons. Moving up the painting, the middle section reveals the outline of three cats stacked on top of each other within the coat. The artist might have chosen to show this through a slightly open coat, or perhaps through the distinctive shapes visible under the fabric. The cats could be of different breeds and colors to add visual interest and humor. The top third of the painting, the upper two feet, features the head of the topmost cat peeking out from the collar of the trenchcoat. The cat might be looking straight ahead, trying to maintain its human disguise, or it might be looking out at the viewer with a humorous or mischievous expression. If the artist chose to include a hat, it would be perched on the cat's head, adding to the disguise. The background of the painting could vary greatly depending on the artist's style and intentions. It could be a simple, solid color to keep the focus on the cats, or it could be a detailed scene that adds to the story - perhaps a city street to tie in with the Vincent Adultman reference, or a domestic setting to contrast with the cats' unusual behavior. Again, this is a hypothetical description and the actual painting could be very different.
- Der_Einzige 3y ago
  Most of what you think is extrapolation is interpolation. Models do NOT extrapolate unless they are given access to resources/data that is not somehow in their training dataset. We have done so much encoding of knowledge that a majority of serious use-cases for LLMs are Interpolation, but the fact that we even need retrieval augmented generation shows that they do not extrapolate.

llogiq 3y ago

Again, there is a lot of words to describe the fact that machine learning is just lossy compression for a bunch of data with the possibility to interpolate between data points and get somewhat plausible results. This means data points may get lost during compression/training, and certain things will look off, whether it be a preference for banana pairs, even numbers of fingers or certain weasel words in verbiage.

bippihippi1 3y ago
all gradient learning creates kernel machines
goldenkey 3y ago
And is RL not? All of these models are constrained by finite weights and then tuned. Are you suggesting we grow a neural network until certain criterion are met with regard to out of distribution test criteria? Hmm
- llogiq 3y ago
  What would that buy you? You train until you push your loss under a certain threshold, then check the external criteria and if they don't hold you train again? Your external criteria would essentially become another part of your loss function, but the whole training would become vastly more inefficient. I'm saying that we shouldn't expect the models to come up with things we didn't train them to come up with.
  - goldenkey 3y ago
    Yeah, it wouldn't buy us anything additional. The only way to really verify would be to look at out of distribution inputs and see if they satisfy what we'd really like. Wouldn't that be true generalization? Otherwise our model architectures or training data is insufficient.

anotheryou 3y ago

Midjourney solution: "--no bunch, two, multiple": https://i.imgur.com/QnJmLr0.jpeg https://i.imgur.com/QnJmLr0.jpeg These models have a tendency to move towards the average, especially if unprompted. As we see here, sometimes even if prompted otherwise. They just have to get better or need a more precise interface like the —no :). We also couldn't have "a man crawling" before and now we can: https://i.imgur.com/ycVpk3i.jpeg https://i.imgur.com/ycVpk3i.jpeg

RugnirViking 3y ago
--no is too strong, it ruins the quality because of negative weight pollution. Something like :: bunch, two, multiple::-0.3 would probably be enough without making them look worse
jonplackett 3y ago
They don't really look like bananas anymore though...
- anotheryou 3y ago
  first one is fine. And I could do it more carefully :) Adjusted and with help of the other commenter°: https://i.imgur.com/e2aE1bQ.jpeg https://i.imgur.com/e2aE1bQ.jpeg And you always have some crap generations :) ° https://news.ycombinator.com/item?id=36584831 https://news.ycombinator.com/item?id=36584831
dafoex 3y ago
I agree. While the article is correct that this problem of subtle bias exists, the solution for most cases is to realise that the computer "does what you say, not what you mean".

tsukikage 3y ago

"Three cats in a trenchcoat" is a good one. Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style. You can get them standing side by side wearing trenchcoats. You can get a cat pyramid or a cat totem or a stack or tower of cats (though often the ability to count to three is then lost). You can't get them to share the trenchcoat. Nothing like that occurred in any training set, and the AIs do not understand spatial relationships between objects ("X is on top of Y, inside Z") so you cannot describe how to arrange the things it does know about in the scene. Dall-E won't do it. Midjourney won't do it. Stable Diffusion won't do it. Eventually, enough images will be seeded into the training sets for this to stop being a useful test. But right now, it gives one a fascinating window on what happens when you try to extrapolate outside the cloud of thingspace described by the training data, rather than just interpolating within it. https://www.toothycat.net/~sham/trenchcoats https://www.toothycat.net/~sham/trenchcoats

goldenkey 3y ago
It's mostly the generation engine, not the logical AI. Here's what GPT-4 says when asked to describe. > As an AI, I can't actually see the painting, but based on the title and the dimensions you've provided, I can imagine a possible description. > The painting, titled "Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style," is a large piece, standing 6 feet tall and 4 feet wide. > At the bottom of the painting, the trenchcoat's hem brushes the ground, obscuring the cats' feet. The trenchcoat is a deep, rich color, perhaps a classic trenchcoat beige or a more dramatic black or navy. The coat is painted with careful detail, showing the texture of the fabric and the gleam of buttons. > Moving up the painting, the middle section reveals the outline of three cats stacked on top of each other within the coat. The artist might have chosen to show this through a slightly open coat, or perhaps through the distinctive shapes visible under the fabric. The cats could be of different breeds and colors to add visual interest and humor. > The top third of the painting, the upper two feet, features the head of the topmost cat peeking out from the collar of the trenchcoat. The cat might be looking straight ahead, trying to maintain its human disguise, or it might be looking out at the viewer with a humorous or mischievous expression. If the artist chose to include a hat, it would be perched on the cat's head, adding to the disguise. > The background of the painting could vary greatly depending on the artist's style and intentions. It could be a simple, solid color to keep the focus on the cats, or it could be a detailed scene that adds to the story - perhaps a city street to tie in with the Vincent Adultman reference, or a domestic setting to contrast with the cats' unusual behavior. > Again, this is a hypothetical description and the actual painting could be very different.

lionkor 3y ago

> Then think of all the wealth disparity that has been introduced into our world. Think of the social anxiety of always being online. Think of the undermining of our democratic institutions. How is this at all whatsoever needed

Joker_vD 3y ago
Well, it's a 2023 piece about AI so of course a paragraph or two of sociological blabbing is required.
Nasrudith 3y ago
It isn't really. But those are the hivemind memes and not agreeing with their bullcrap gets you looked at like you are a crazy person or worse, not part of the in crowd. High school has done some serious and lasting damage to socialization throughout the world as social backbiting popularity bullshit became the norm. Not to mention complaining about wealth disparity from investment as if it is a sin of the technology is asinine. If money can produce better results you will see wealth disparity from the simple feedback loop of "more money -> better results -> more money". Your neighbor taking up jogging while you sit on the couch all day is increasing a health disparity, should your neighbor stop jogging then? Undermining of democratic institutions is just plain a dumb meme from vested interests who cast what were previously abuses or necessary evils of the system at best as integral to democracy. We would all recognize blaming the media for undermining the Catholic Church by documenting their child sexual abuse as batshit insane. Yet somehow it is different when it comes to newspapers moaning about their loss of monopolies and influence of cliques.

avidphantasm 3y ago

[flagged]

frankreyes 3y ago

LLMs don't understand meaning, they're just a statistical random number generator.

dr_hooo 3y ago
You're just a statistical number generator...

RugnirViking 3y ago

If you want an imo more interesting example try "bulldozer". It's missing a quite... Important piece to our conception of those machines

lordnacho 3y ago

LLM doesn't know stuff, it has few models beyond an extremely deep association between (in this case) descriptions and pictures. Humans have shitty models for stuff, but we have them. We lack the massively deep numerical associations. It can't be that long before someone makes an AI that says knows a few common models like "stuff made of steel is rigid, stuff made of cloth is soft, etc". Along with "If someone keeps emphasizing a number, that's the number of things he wants".

im3w1l 3y ago
Me: A designer has created two artpieces. The first one "The gattoya" is made of steel bars welded into a spiral shape. The second "Misiblur" is made of cloth and somewhat resembles a butterfly. Which is most comfortable to rest your head on and why? Chatgpt: Based on the description provided, it is likely that "Misiblur," the art piece made of cloth resembling a butterfly, would be more comfortable to rest your head on compared to "The gattoya," which is made of steel bars welded into a spiral shape. The reason for this is that cloth generally has a softer and more cushioned surface compared to steel bars. When resting your head on Misiblur, the fabric would provide a certain level of support and padding, which could make it more comfortable for resting your head. On the other hand, resting your head on steel bars, as in the case of The gattoya, would likely be less comfortable due to the hardness and lack of cushioning provided by the material. Steel bars are not designed to be soft or ergonomic for resting purposes, so it may not provide the desired comfort. It's important to note that the comfort level can vary depending on individual preferences and the specific design and construction of the art pieces. However, based on the given materials and descriptions, Misiblur made of cloth would generally be the more comfortable option for resting your head.
- lordnacho 3y ago
  Yes but that's text, and there's plenty of text that will tell you that steel is rigid, and rigid is uncomfortable. Also you gave it an A/B choice, random chance is 50%. Does it insist it's right if you prompt it the other way? Does it make pictures of people who are comfortable in one but not the other? Most of the time when I see something funny in LLMs it's in the images. People with 8 fingers, elbows bent backwards, that kind of thing. Spatial models.

sarabande 3y ago

Here is a summary, part ChatGPT and part me: The “Lone Banana Problem” describes subtle biases of Large Language Models (LLMs) in AI: LLMs reproduce the statistical average of the inputs that they have consumed in the context of the question they have been asked. It's called that problem because the model used to generate images has never seen an individual banana, so when prompted always generates two bananas.

bugmen0t 3y ago

Would be kinda funny if the lone bananas have been removed from the training dataset because they are not part of the depicted item but are merely in the picture for scale :)

maltelandwehr 3y ago

„A single banana —-v 5.2“ in Midjourney gives me flur pictures. Two of these pictures only contain a single banana.

__s 3y ago

They didn't try asking for half a banana?

IIAOPSW 3y ago

or it was always just a dice roll, and asking the same prompts again will give you sometimes 1, usually 2, sometimes 3 bananas.

FrustratedMonky 3y ago

Whatever the critiques of length, this is a good write up on 'bias', that is not about race or politics, and thus a good neutral example about how training sets can skew results in un-intended ways. I didn't think it was that long, it might be that some are cherry picking the points they find interesting and think the rest could have been edited out. I could have stood even more discussion on how this 'blind spot' in the AI model is very much akin to our own blind spots.

GuB-42 3y ago

I remember about 10 years ago in the context of using neural network for image classification, they had a study of what the AI actually saw that justified its decision. This, by the way, is how we got "Deep Dream". For "dumbbell", for the AI, no dumbbell was complete without a muscular arm holding it. That's because in most images in the training dataset have the dumbbells being held, so the system integrated the arm in the pattern. I guess it is the same idea here, most pictures of bananas show several of them, so for the AI, bananas are things that don't go alone.

eimrine 3y ago

I used to experience a very similar problem [1]. They say NNs are bad at counting. But at least your picture has kind of a single piece of bananas, my problem remained unsolved. https://news.ycombinator.com/item?id=32875215 https://news.ycombinator.com/item?id=32875215

theknocker 3y ago

[dead]

crosen99 3y ago

> AIs, at their current level of development, don’t perceive objects in the way that we do – they understand commonly occurring patterns. You see this claim everywhere - that AI operates on statistics and patterns and not actual understanding. But human understanding is entirely about statistics and patterns. When a human sees a collection of particles and recognizes it as, say, a car, all they are doing is recognizing the car-like patterns in how the particles are organized that have a strong statistical correlation with prior observations of things classified as a car. Am I missing something?

lamontcg 3y ago
We're not (initially anyway) trained on photography and literature and reddit, our first experience of a banana is probably eating one.
lolc 3y ago
After a decade of billions invested in systems with models trained for object recognition in traffic, these models still struggle with object permanence. Why should we expect some rando painter models to do better? They'll paint a lamppost through a car and not notice anything wrong with it. The day where these models can show shoes under a fence and a reflection of the person behind the fence in an opposing shop window? That day will come. But not on the current crop.

BryanLegend 3y ago

I unsubscribed from Midjourney because I couldn't direct it to place an object in a specific place. It's impossible to do something like, "A sphere on the left of the picture." and have it understand it. After having iterative conversations in ChatGPT, Midjourney felt like it has an extremely poor grasp of language.

chaosjevil 3y ago

TL;DR: AI doesn't understand human language prompts, it's just associating specific tokens with specific outputs.

13years 3y ago

At Digital Science, we believe that we have a responsibility to ensure that the technologies that we release are well tested and well understood. The use cases where we deploy AI have to be appropriate for the level at which we know the AI can perform and any functionality needs to come with a “health warning” so that people know what they need to look for – when they can trust an AI and when they shouldn’t. We don't even understand ourselves and we hope to model AI alignment in some image of humanity with the goal that it will be just as benevolent as our fractured war eager society. Yes it is a paradox indeed. I submit there is likely a limitation to how much theoretically AI could ever improve beyond ourselves in the regards to bias. I've described this as the AI Bias Paradox - https://www.mindprison.cc/p/ai-the-bias-paradox https://www.mindprison.cc/p/ai-the-bias-paradox

deleted 3y ago

[deleted]

Der_Einzige 3y ago

This isn't a real problem anymore. Composable Diffusion, Regional Prompting, and Controlnet literally solved all of these issues. The author shows what the world is learning, "Johnny can't prompt", and those who learn how to use Generative AI well are going to be continuing to propel up in their careers while the general public incorrectly concludes that the tools "don't work".

gweinberg 3y ago

It's really weird that the monkey is eating two bananas at once, I would think in training data monkeys would almost always be eating one banana at a time. I've been told that in the real world monkeys and chimps don't peel bananas when they eat them, they eat the peel and all, but i don't know if it's true. Whatever happens in the real world, drawings always show monkeys peeling the bananas as they eat them and I would expect a prompt "show a monkey eating a banana" to show it eating a single peeled banana.