4 ms·

Hi, author here. This article was many years in the making. It's ostensibly a story about reading minds but really it's about the unreasonable power of high-dim

Hi, author here. This article was many years in the making. It's ostensibly a story about reading minds but really it's about the unreasonable power of high-dimensional vector spaces. That made it pretty tough to write: how do you explain dimensionality reduction, PCA, word2vec, etc., and the wonders of high-dimensional "embeddings" (of the sort you find in deep neural nets) when a lot—or all—of these ideas might be new to the reader? I'm not sure—but this was my attempt!

bertil 5y ago
Great article! I’m developing Machine learning systems and my partner is working on psychiatric use of Deep-Brain stimulation, so a rare moment that we can share. Very minor point: the King - male + female = Queen is a good example, but widely decried as not true by specialists. I don’t have much better examples (I haven’t been able to tell if Paris - France + England = London, for instance) but if you reuse that story, it makes sense to investigate that myth. There’s a lot there too.
Jun8 5y ago
Great article! I think you’ve done a great job of introducing these difficult concepts in simple language. I saved to Pocket and it’s already got a Best Of label there. PCA (KLT) can be introduced as a generalization of the Fourier Transform. This can follow from using a cocktail mix analogy to Fourier Series. When I was a TA this was the approach I took with students, which seemed to make things easier for them. An introductory post on PCA vs FA is here: https://towardsdatascience.com/what-is-the-difference-between-pca-and-factor-analysis-5362ef6fa6f9 https://towardsdatascience.com/what-is-the-difference-betwee... Personal note: Susan Dumais, mentioned in the article also did great early work in text summarization, just after she joined Microsoft. I tried using some of her approached in video summarization in my PHDin early 2000s. How time flies.
- ReaLNero 5y ago
  PCA applies an orthogonal linear transformation, while FA uses a series of coefficients to scale a sequence of functions, which are then integrated. They are similar in use but very different in method. Calling one a generalization of the other seems misguided?
  - aaaaaaaaaaab 5y ago
    Ummm… the Fourier Transform is an orthonormal linear transformation.
- canjobear 5y ago
  I don’t think explaining it as a generalization of the Fourier transform is going to help very much with New Yorker readers.
boldslogan 5y ago
Hey I liked all the examples. Thought you might like this one: A Geometric Analysis of Five Moral Principles (OUP 2017) Ethics using vectors or from a description of the technique: The geometric approach derives its normative force from the Aristotelian dictum that we should “treat like cases alike.” The more similar a pair of cases are, the more reason do we have to treat the cases alike. These similarity relations can be analyzed and represented geometrically. In such a geometric representation, the distance in moral space between cases reflects their degree of similarity. The more similar a pair of cases are from a moral point of view, the shorter is the distance between them.
mvkel 5y ago
It’s cool to see you here! Fascinating article. I had no idea this was being pursued in an applied way; assumed it was all theoretical. Exciting!
anigbrowl 5y ago
I think you did a very good job - it captured the feeling that it's almost sorcery which still hits me any time I successfully apply it, without getting bogged down in technicalities. I think it's OK to be superficial as long as you give people enough information to look up and learn more about it. Mentioning word2vec will certainly give interested readers a head start.
a-dub 5y ago
i find the simplest way to explain pca to a general audience is to draw a ellipse of points off center and tilted in 3d space, and then draw plots for x, y and z. then center the ellipse and rotate the axes to match the major and minor axes of the ellipse and then show how it can be drawn in just x and y and that those x and y plots are far easier to interpret. done.
Hizonner 5y ago
Why did you write an article glorifying people who are working as hard as they can toward dystopia?
- ausbah 5y ago
  why are you asking a loaded question?
  - MisterTea 5y ago
    - dang 5y ago
      Maybe so, but please don't post unsubstantive comments here.
      - MisterTea 5y ago
        "unsubstantive"? This technology will be used against people. I just wanted to point it out in the most unflattering way possible. Like graphic warning and danger signs, it makes people stop and think. And strange to hear from you for a second time in the last week or so.
        dang 5y ago
        Of course it's unsubstantive. It's not just an internet cliché, it's nowhere near the top of the internet cliché barrel. If you have something interesting to say, please say it explicitly, without tedious tropes, and without flamebait. > And strange to hear from you for a second time in the last week or so. I had no idea I'd replied to you repeatedly, but if so, the simplest explanation is that you've been breaking the site guidelines repeatedly. It would be helpful if you'd review them and stick to them from now on: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
  - revolvingocelot 5y ago
enugu 5y ago
That was a fascinating article, I liked how you covered the human element in how it helps the paralyzed and the intuitions/visuals of the researchers in the field. Future applications can be good or bad, but of course that makes it even more important to record the early history of the field and these kind of articles will also help in start the ethics discussion at an earlier stage.

bertil 5y ago

Great article! I’m developing Machine learning systems and my partner is working on psychiatric use of Deep-Brain stimulation, so a rare moment that we can share. Very minor point: the King - male + female = Queen is a good example, but widely decried as not true by specialists. I don’t have much better examples (I haven’t been able to tell if Paris - France + England = London, for instance) but if you reuse that story, it makes sense to investigate that myth. There’s a lot there too.

Jun8 5y ago

Great article! I think you’ve done a great job of introducing these difficult concepts in simple language. I saved to Pocket and it’s already got a Best Of label there. PCA (KLT) can be introduced as a generalization of the Fourier Transform. This can follow from using a cocktail mix analogy to Fourier Series. When I was a TA this was the approach I took with students, which seemed to make things easier for them. An introductory post on PCA vs FA is here: https://towardsdatascience.com/what-is-the-difference-between-pca-and-factor-analysis-5362ef6fa6f9 https://towardsdatascience.com/what-is-the-difference-betwee... Personal note: Susan Dumais, mentioned in the article also did great early work in text summarization, just after she joined Microsoft. I tried using some of her approached in video summarization in my PHDin early 2000s. How time flies.

ReaLNero 5y ago
PCA applies an orthogonal linear transformation, while FA uses a series of coefficients to scale a sequence of functions, which are then integrated. They are similar in use but very different in method. Calling one a generalization of the other seems misguided?
- aaaaaaaaaaab 5y ago
  Ummm… the Fourier Transform is an orthonormal linear transformation.
canjobear 5y ago
I don’t think explaining it as a generalization of the Fourier transform is going to help very much with New Yorker readers.

boldslogan 5y ago

Hey I liked all the examples. Thought you might like this one: A Geometric Analysis of Five Moral Principles (OUP 2017) Ethics using vectors or from a description of the technique: The geometric approach derives its normative force from the Aristotelian dictum that we should “treat like cases alike.” The more similar a pair of cases are, the more reason do we have to treat the cases alike. These similarity relations can be analyzed and represented geometrically. In such a geometric representation, the distance in moral space between cases reflects their degree of similarity. The more similar a pair of cases are from a moral point of view, the shorter is the distance between them.

mvkel 5y ago

It’s cool to see you here! Fascinating article. I had no idea this was being pursued in an applied way; assumed it was all theoretical. Exciting!

anigbrowl 5y ago

I think you did a very good job - it captured the feeling that it's almost sorcery which still hits me any time I successfully apply it, without getting bogged down in technicalities. I think it's OK to be superficial as long as you give people enough information to look up and learn more about it. Mentioning word2vec will certainly give interested readers a head start.

a-dub 5y ago

i find the simplest way to explain pca to a general audience is to draw a ellipse of points off center and tilted in 3d space, and then draw plots for x, y and z. then center the ellipse and rotate the axes to match the major and minor axes of the ellipse and then show how it can be drawn in just x and y and that those x and y plots are far easier to interpret. done.

Hizonner 5y ago

Why did you write an article glorifying people who are working as hard as they can toward dystopia?

ausbah 5y ago
why are you asking a loaded question?
- MisterTea 5y ago
  - dang 5y ago
    Maybe so, but please don't post unsubstantive comments here.
    - MisterTea 5y ago
      "unsubstantive"? This technology will be used against people. I just wanted to point it out in the most unflattering way possible. Like graphic warning and danger signs, it makes people stop and think. And strange to hear from you for a second time in the last week or so.
      - dang 5y ago
        Of course it's unsubstantive. It's not just an internet cliché, it's nowhere near the top of the internet cliché barrel. If you have something interesting to say, please say it explicitly, without tedious tropes, and without flamebait. > And strange to hear from you for a second time in the last week or so. I had no idea I'd replied to you repeatedly, but if so, the simplest explanation is that you've been breaking the site guidelines repeatedly. It would be helpful if you'd review them and stick to them from now on: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
- revolvingocelot 5y ago

enugu 5y ago

That was a fascinating article, I liked how you covered the human element in how it helps the paralyzed and the intuitions/visuals of the researchers in the field. Future applications can be good or bad, but of course that makes it even more important to record the early history of the field and these kind of articles will also help in start the ethics discussion at an earlier stage.