5 ms·

The voyage looks somewhat random, yet inspiring. I can imagine using it to grab ideas for the overarching story arc in some storytelling. > I am waiting at the

The voyage looks somewhat random, yet inspiring. I can imagine using it to grab ideas for the overarching story arc in some storytelling. > I am waiting at the entrance of the fantasy kingdom. ... > I have a revelation about reality. Main problem is, it doesn't look at all robust. A small change in a single word at the start or end sentence produces a whole different result. That doesn't look like a "space" at all, as insights about closeness do not work in it.

TuringTest 9y ago
Ok, I've played with the "Look around" example and it does behave as a space, with the "Closer to home" option producing somewhat similar results to the starting point, and the slider producing concepts that look farther away from it. Maybe the "Gimme a gradient" option step is too large for sentences that are far away, and an option to produce more steps, closer between them, would produce more intuitive results?
OscarCunningham 9y ago
I guess that can happen sometimes in actual spaces too. The shortest path from a point near the north pole to the south pole can change dramatically if you slightly alter its endpoints. Given a high-dimensional space it could be that most possible paths behave in this weird way.
- jimmytidey 9y ago
  How useful is the spatial analogy if the space in question defies all our spatial intuitions? I can see it has mathematical meaning, but isn't the whole post about having intuitive understanding of sentence space?
  - gugagore 9y ago
    I think it's less about intuitive understanding, and more about having a flat, numerical representation of a sentence that (qualitatively) satisfies some properties.
ppod 9y ago
This is essentially the reason that differentiable methods alone don't work that well for NLP. Language is a discrete arbitrary symbol system, so small changes may have large effects at many levels. ("Dog bites man" is not like "Man bites dog", discrete is not like discreet), etc.
maljx 9y ago
I feel like the "middle" sentence is not intuitively what you'd normally think of something in between. Even for sentences that are quite close. "Eve walked home slowly" -> "Eve walked home quickly" gives a middle result of "As white hand smiled."
rockostrich 9y ago
Robin is an author that dabbles in machine learning and farming olives for making olive oil. It's not surprising that his examples aren't robust, but I always enjoy reading about his endeavors because I end up in a rabbit hole on whatever subject he's writing about.

TuringTest 9y ago

Ok, I've played with the "Look around" example and it does behave as a space, with the "Closer to home" option producing somewhat similar results to the starting point, and the slider producing concepts that look farther away from it. Maybe the "Gimme a gradient" option step is too large for sentences that are far away, and an option to produce more steps, closer between them, would produce more intuitive results?

OscarCunningham 9y ago

I guess that can happen sometimes in actual spaces too. The shortest path from a point near the north pole to the south pole can change dramatically if you slightly alter its endpoints. Given a high-dimensional space it could be that most possible paths behave in this weird way.

jimmytidey 9y ago
How useful is the spatial analogy if the space in question defies all our spatial intuitions? I can see it has mathematical meaning, but isn't the whole post about having intuitive understanding of sentence space?
- gugagore 9y ago
  I think it's less about intuitive understanding, and more about having a flat, numerical representation of a sentence that (qualitatively) satisfies some properties.

ppod 9y ago

This is essentially the reason that differentiable methods alone don't work that well for NLP. Language is a discrete arbitrary symbol system, so small changes may have large effects at many levels. ("Dog bites man" is not like "Man bites dog", discrete is not like discreet), etc.

maljx 9y ago

I feel like the "middle" sentence is not intuitively what you'd normally think of something in between. Even for sentences that are quite close. "Eve walked home slowly" -> "Eve walked home quickly" gives a middle result of "As white hand smiled."

rockostrich 9y ago

Robin is an author that dabbles in machine learning and farming olives for making olive oil. It's not surprising that his examples aren't robust, but I always enjoy reading about his endeavors because I end up in a rabbit hole on whatever subject he's writing about.