6 ms·

Some unlinked features... If you put the Dewey division in the URL, the directory auto-opens. e.g. here are episodes about prehistoric life (my current jumping

by genmon 4y ago

Some unlinked features... If you put the Dewey division in the URL, the directory auto-opens. e.g. here are episodes about prehistoric life (my current jumping-off point) https://genmon.github.io/braggoscope/directory#560 https://genmon.github.io/braggoscope/directory#560 There's a visual map of episodes. After principal component analysis of the episode embedding vectors, these are the most significant two components as the x,y https://genmon.github.io/braggoscope/map.html https://genmon.github.io/braggoscope/map.html (it's not super useful tbh -- e.g. the Manhattan Project and the Cambrian Explosion have the same x,y... presumably because they are both about explosions?) Many episodes have a reading list, and these are all linked to Google Books (so you can purchase/check out from a library), e.g. this episode page https://genmon.github.io/braggoscope/2022/10/20/the-fishtetrapod-transition.html https://genmon.github.io/braggoscope/2022/10/20/the-fishtetr... There are ~4,600 books, and I have ~88% coverage on getting a Google Books page from the original data. Any ideas about what to do with this big list of academic-recommended books v welcome!

p0pcult 4y ago
Love the visual map. What does color mean? Any way to do a 3rd PC, and put the visualization in a cube one can toy around with?
- genmon 4y ago
  Colour is the 3rd component -- I wanted to see the difference between overlapping episodes. As for the 3D plot... here you go! https://interconnected.org/more/2023/03/in_our_time-PCA-3D-plot.html https://interconnected.org/more/2023/03/in_our_time-PCA-3D-p... Basic PCA + Plotly is actually in OpenAI's official Python library (in `embedding_utils`) -- this plot is just the output from that.
  - p0pcult 4y ago
    :D this made my day.
CrypticShift 4y ago
Excellent! I'd love to see a script that sends a list of "descriptions" (1-100 words) to ChatGPT and directly gives you back a ready-made (embedding vectors closeness) map in a (textual) graph/chart format (like your above map or your plot https://interconnected.org/more/2023/02/in_our_time-PCA-plot.html https://interconnected.org/more/2023/02/in_our_time-PCA-plot...)
- genmon 4y ago
  It turns out that "closeness" is usually hard to visualise/explore when you're dealing with a 1,000-dimensional space... and PCA has the failures mentioned above. It's weird -- it's locally useful to navigate, and at a high level kinda useful, but only if you squint and don't look at the problems. So I feel like a fisheye visualisation would be appropriate? That's something that I'm exploring in other projects.
  - gwern 4y ago
    I wouldn't necessarily reach for PCA. No reason to think that the first two principal components necessarily encode anything particularly interesting. If you want to lay out each point in 2D in a way which keeps similar points nearby, something like t-SNE is worth a try - visualizing embeddings is what it was invented for.
    - genmon 4y ago
      Excellent, new to me and I'll give it a go, thanks! I gravitate to PCA for terrible reasons (undergrad so it's what I think of first) and like you say, it's beguiling yet disappointing, the components rarely have any human meaning.
  - pigscantfly 4y ago
    I'd suggest trying t-SNE [1] instead; you'll be losing almost all of the variance by projecting onto the first two eigenvectors produced by PCA. [1] http://karpathy.github.io/2014/07/02/visualizing-top-tweeps-with-t-sne-in-Javascript/ http://karpathy.github.io/2014/07/02/visualizing-top-tweeps-...
sacrosancty 4y ago
Your PCA is awesome. The horizontal axis seem to go from people (high level nature?) on the left to physics (low level nature?) on the right, while the vertical axis seems to go from individuals and particles (small things?) at the bottom, up to civilizations and the universe (big things?) at the top. Maybe the Manhattan Project and Cambrian Period (not explosion) are together because they're both big things in their fields and physics and evolution are near each other horizontally?
flir 4y ago
This is great. I'll certainly be thinking of "classification" uses for ChatGPT in the future. Thinking out loud: Add the experts, not just the reading lists. They're a jumping-off point into academic-paper-space. What have they published? In what journals? Who have they collaborated with?
- fnordpiglet 4y ago
  Here’s a powerful use - content moderation. Today we literally traumatize content moderators with the dregs of the human mind. Chatgpt is fairly good at identifying the classification of content on many dimensions, including the ones it’s actively screens for. Regardless of how you personally feel about content moderation, I would be happy to see humans not have to be actively involved in it and face the traumas they must live with for where the moderation happens. I’m sure it’ll get things wrong, but humans do too.
  - chezelenkoooo 4y ago
    I'm sure I remember Zuckerberg talking about this very thing in one of the Congress interviews he had. I don't disagree with you, the content producers are always going to outnumber the moderators by a massive margin. It makes reasonable moderation very difficult.
  - nl 4y ago
    OpenAI have a content moderation endpoint for this too.
  - snowe2010 4y ago
    It's good at it because OpenAI outsources the bad stuff to Kenya https://time.com/6247678/openai-chatgpt-kenya-workers/ https://time.com/6247678/openai-chatgpt-kenya-workers/ People still have to look at this horrendous stuff.
    - fnordpiglet 4y ago
      To bootstrap, yes. But once you’ve built the machine, the people aren’t necessary.
Tarq0n 4y ago
Love the plot, for high dimensional embeddings try UMAP for a better result, maybe T-sne too if you're interested in clusters.
shellac 4y ago
> https://genmon.github.io/braggoscope/directory#560 https://genmon.github.io/braggoscope/directory#560 Shouldn't https://www.bbc.co.uk/programmes/b00lh2s3 https://www.bbc.co.uk/programmes/b00lh2s3 (Ediacara Biota) be in there?

by genmon 4y ago

p0pcult 4y ago

Love the visual map. What does color mean? Any way to do a 3rd PC, and put the visualization in a cube one can toy around with?

genmon 4y ago
Colour is the 3rd component -- I wanted to see the difference between overlapping episodes. As for the 3D plot... here you go! https://interconnected.org/more/2023/03/in_our_time-PCA-3D-plot.html https://interconnected.org/more/2023/03/in_our_time-PCA-3D-p... Basic PCA + Plotly is actually in OpenAI's official Python library (in `embedding_utils`) -- this plot is just the output from that.
- p0pcult 4y ago
  :D this made my day.

CrypticShift 4y ago

Excellent! I'd love to see a script that sends a list of "descriptions" (1-100 words) to ChatGPT and directly gives you back a ready-made (embedding vectors closeness) map in a (textual) graph/chart format (like your above map or your plot https://interconnected.org/more/2023/02/in_our_time-PCA-plot.html https://interconnected.org/more/2023/02/in_our_time-PCA-plot...)

genmon 4y ago
It turns out that "closeness" is usually hard to visualise/explore when you're dealing with a 1,000-dimensional space... and PCA has the failures mentioned above. It's weird -- it's locally useful to navigate, and at a high level kinda useful, but only if you squint and don't look at the problems. So I feel like a fisheye visualisation would be appropriate? That's something that I'm exploring in other projects.
- gwern 4y ago
  I wouldn't necessarily reach for PCA. No reason to think that the first two principal components necessarily encode anything particularly interesting. If you want to lay out each point in 2D in a way which keeps similar points nearby, something like t-SNE is worth a try - visualizing embeddings is what it was invented for.
  - genmon 4y ago
    Excellent, new to me and I'll give it a go, thanks! I gravitate to PCA for terrible reasons (undergrad so it's what I think of first) and like you say, it's beguiling yet disappointing, the components rarely have any human meaning.
- pigscantfly 4y ago
  I'd suggest trying t-SNE [1] instead; you'll be losing almost all of the variance by projecting onto the first two eigenvectors produced by PCA. [1] http://karpathy.github.io/2014/07/02/visualizing-top-tweeps-with-t-sne-in-Javascript/ http://karpathy.github.io/2014/07/02/visualizing-top-tweeps-...

sacrosancty 4y ago

Your PCA is awesome. The horizontal axis seem to go from people (high level nature?) on the left to physics (low level nature?) on the right, while the vertical axis seems to go from individuals and particles (small things?) at the bottom, up to civilizations and the universe (big things?) at the top. Maybe the Manhattan Project and Cambrian Period (not explosion) are together because they're both big things in their fields and physics and evolution are near each other horizontally?

flir 4y ago

This is great. I'll certainly be thinking of "classification" uses for ChatGPT in the future. Thinking out loud: Add the experts, not just the reading lists. They're a jumping-off point into academic-paper-space. What have they published? In what journals? Who have they collaborated with?

fnordpiglet 4y ago
Here’s a powerful use - content moderation. Today we literally traumatize content moderators with the dregs of the human mind. Chatgpt is fairly good at identifying the classification of content on many dimensions, including the ones it’s actively screens for. Regardless of how you personally feel about content moderation, I would be happy to see humans not have to be actively involved in it and face the traumas they must live with for where the moderation happens. I’m sure it’ll get things wrong, but humans do too.
- chezelenkoooo 4y ago
  I'm sure I remember Zuckerberg talking about this very thing in one of the Congress interviews he had. I don't disagree with you, the content producers are always going to outnumber the moderators by a massive margin. It makes reasonable moderation very difficult.
- nl 4y ago
  OpenAI have a content moderation endpoint for this too.
- snowe2010 4y ago
  It's good at it because OpenAI outsources the bad stuff to Kenya https://time.com/6247678/openai-chatgpt-kenya-workers/ https://time.com/6247678/openai-chatgpt-kenya-workers/ People still have to look at this horrendous stuff.
  - fnordpiglet 4y ago
    To bootstrap, yes. But once you’ve built the machine, the people aren’t necessary.

Tarq0n 4y ago

Love the plot, for high dimensional embeddings try UMAP for a better result, maybe T-sne too if you're interested in clusters.

shellac 4y ago

> https://genmon.github.io/braggoscope/directory#560 https://genmon.github.io/braggoscope/directory#560 Shouldn't https://www.bbc.co.uk/programmes/b00lh2s3 https://www.bbc.co.uk/programmes/b00lh2s3 (Ediacara Biota) be in there?