7 ms·

Here comes the Muybridge camera moment but for text

249 points by RA2lover 2y ago

082349872349872 2y ago
> What would it mean to listen to a politician speak on TV, and in real-time see a rhetorical manoeuvre that masks a persuasive bait and switch? Why do I suspect the offence will always be ahead of the defence in these areas? I'd earlier suggested that everyone, in elementary school, ought to watch Ancient Aliens and attempt to note the moment where each episode jumps the shark. I take it we could attempt this with LLMs, now?
- rablackburn 2y ago
  > Why do I suspect the offence will always be ahead of the defence in these areas? because destroying is easier than creating/entropy increases over time? The only solution I can see is working on turning bad actors into good actors, or another way: positive reinforcement cycles. No idea what that would look like with regard to LLMs though.
  - pixl97 2y ago
    At the end of the day there is no permanent solution. In nature we typically don't see something 'win' and that's the end of the story. I mean yes things do go extinct, but the winner always has something new to deal with. Could be a more advanced predator eating all it's food sources. Could be a bacteria that it's not resistant to. Simply put, when there's entropy on the table, something is going to evolve to take it with the least amount of work possible.
dhosek 2y ago
For those perplexed by the headline, the Muybridge camera moment refers to Eadweard Muybridge who managed via camera photos taken in rapid succession to prove that when a horse runs it at times has all four legs above the ground. https://en.wikipedia.org/wiki/Eadweard_Muybridge https://en.wikipedia.org/wiki/Eadweard_Muybridge (the article doesn’t bother to mention any of this until near the end in the tl;dr section, which since it’s tl and you dr, you never got to).
- stavros 2y ago
  Not only that, but the tldr basically only talks about that, so it's not much of a summary at all. I read the tldr and I have no idea what the article is about.
- Animats 2y ago
  (On an irrelevant note, the Stanford Barn, where those pictures were taken, has gradually been closed off to the world. It was open to the public until COVID. It's still there, and there's a Stanford equestrian team, but road access has been cut and all mentions of the barn removed from directional signs.)
  - gausswho 2y ago
    There are so many of these places I've encountered what used to be publicly available pre-COVID and are no longer. The reasons/excuses vary. Example: Sometimes it's a symptom of a small business already wanted a reason to pivot to a new venture, and they keep the old thing going to profit from some old whales while in transition.
    - dhosek 2y ago
      There was a lot of that post 9/11 too. It used to be that you could walk into nearly any office building in the world with little more than a smile and a confident wave. A lot of previously public areas got locked down on September 12th.
      - PopAlongKid 2y ago
        Office building security changed significantly much earlier than 2001. The mass shooting in 1993 at 101 California Street in San Francisco was the beginning of many such changes. The attack [...] also precipitated sweeping changes in downtown San Francisco. Before Ferri walked into the building that July day, almost no high-rises in the city had security measures. While many had a front desk, only a handful checked badges. The building at 101 California had two side entrances that were completely unguarded. The Examiner reported that at the time, the Chevron building and Charles Schwab’s SF headquarters had the toughest security in town; electronic badges were required at Chevron, an anomaly in 1993. Today, security checks are standard at offices large and small, a fundamental shift that happened because of 101 California. https://www.police1.com/active-shooter/articles/101-california-the-high-rise-shooting-that-changed-san-francisco-95NXjtYNeX3sBOFn/ https://www.police1.com/active-shooter/articles/101-californ...
qup 2y ago
https://archive.is/EcQfE https://archive.is/EcQfE Site is struggling
anigbrowl 2y ago
Zardoz predicted this ~50 years ago
nickreese 2y ago
I thoroughly enjoyed reading this style of loose connected thoughts.
kepano 2y ago
The repercussions of what the author summarizes as "could you colour-grade a book?" still feel wildly unknown to me, even after a couple years of thinking about it (see Photoshop for text [1][2]). Partially it's because we're still wrapping our heads around what kind of experience this might enable. The tools still feel ahead of the medium. I think we're closer to Niépce than Muybridge. In photography terms, we've just figured out how to capture photons on paper — and artists haven't figured out how to use that to make something interesting. [1] https://news.ycombinator.com/item?id=33253606 https://news.ycombinator.com/item?id=33253606 [2] https://stephango.com/photoshop-for-text https://stephango.com/photoshop-for-text
- throw46365 2y ago
  > The tools still feel ahead of the medium. Or maybe it's that we instinctively feel that writing should still be linear writing, if reading is still going to be linear reading. Personally I think the "photoshop for text" analogy shows just how misguided it is to expect people to tolerate words that were calculated, not crafted. Literacy is too important to mess with like this.
  - kepano 2y ago
    Genuine question — do you think synthetic images pose less of a problem than synthetic text? If yes, why?
    - throw46365 2y ago
      Images — photos, paintings, designs — are not primary human expression. Words are fundamental, dense, often objectively chosen, and the most primary way of communicating thoughts. Asking someone to read your thoughts that you didn’t actually even think, because you’d rather save the time writing them, is profoundly disrespectful to the reader, who has to invest the same amount of time reading generated words as real ones. Which is not to say that I think passing off generative images as one’s own work is not disrespectful. Or that extensive, unreal body sculpting or skin retouching is not — as a photographer I believe that to also often be not just unethical but immoral. But a judgement on a retouched image is less of a burden of time. I would likely judge someone who uses ChatGPT to communicate personally with me as harshly as I would judge them editing a photo to deliberately lie to me. (Which is not to say that I don’t think GPTs have inherent grammatical advantages for cleaning up poorly-written text; I do think generating entirely new text is disrespectful to the reader, though)
Animats 2y ago
So embedding space itself is interesting. It's more than a step to an LLM. That's been known for a while, back to that early result where "King" - "Man" + "Woman" -> "Queen". This article, though, suggests more uses for embedding spaces. This could be interesting. It's a step beyond viewing them as a black box.
- 082349872349872 2y ago
  Is ♔ - m + f = ♕ specific to embeddings, or does it also work in https://en.wikipedia.org/wiki/Formal_concept_analysis#Example https://en.wikipedia.org/wiki/Formal_concept_analysis#Exampl... ? (either as ♔ ⊕ f ⊕ m = ♕ or as ♔ ⋀ not(m) ⋁ f = ♕?) [alas, HN scrubs venus and mars symbols, and I shall spare you all the ancient egyptian hieroglyphs and O'Keeffean mathematical symbols, so `f` and `m` they are]
szvsw 2y ago
One thing I always find interesting but not discussed all that much at least in things I’ve read is - what happens in the spaces between the data? Obviously this is an incredibly high dimensional space which is only sparsely populated by the entirety of the English language; all tokens, etc. if the space is truly structured well enough, then there is a huge amount of interesting, implicit, almost platonic meaning occurring in the spaces between the data - synthetic? Dialectic? Idk. Anyways, I think those areas are a space that algorithmic intelligence will be able to develop its own notions of semantics and creativity in expression. Things that might typically be ineffable may find easy expression somewhere in embedding space. Heidegger’s thisness might be easily located somewhere in a latent representation… this is probably some linguistics 101 stuff but it’s still fascinating imo.
- Der_Einzige 2y ago
  Ya I'm having my return to plato moment. It really feels like we are the dēmiurgós right now with AI systems. The nature of interpolation vs extrapolation and the exploration of latent spaces will answer a lot of philosophical questions that we didn't expect to be answered so quickly, and by computers of all things.
- skydhash 2y ago
  I strongly believe there's nothing there other than gibberish. Piping /dev/random to a word selector will probably enumerates everything inside that set. There's a reason we can translate between every language on earth. That's because it's the same earth and reality. So there's a common sets of concepts that gives us the foundational rules of languages. Which is the data that you're speaking about.
- mortenjorck 2y ago
  Now this is a fun idea. If you think of embeddings as a sort of quantization of latent space, what would happen if you “turned off” that quantization? It would obviously make no sense to us, as we can only understand the output of vectors that map to tokens in languages we speak, but you could imagine a language model writing something in a sort of platonic, infinitely precise language that another model with the same latent space could then interpret.
zharknado 2y ago
> Could you dynamically change the register or tone of text depending on audience, or the reading age, or dial up the formality or subjective examples or mentions of wildlife, depending on the psychological fingerprint of the reader or listener? This seems plausible, and amazing or terrible depending on the application. An amazing application would be textbooks that adapt to use examples, analogies, pacing, etc. that enhance the reader’s engagement and understanding. An unfortunate application would be mapping which features are persuasive to individual users for hyper-targeted advertising and propaganda. A terrible application would be tracking latent political dissent to punish people for thought-crime.
- lsaferite 2y ago
  I'm sure it comes up frequently, but the adapting textbook thought reminds me of the "Young Lady's Illustrated Primer" from Diamond Age.
sebmellen 2y ago
Terence McKenna phrased this wonderfully, by saying “It seems to me that language is some kind of enterprise of human beings that is not finished.” The full quote is more psychedelic, in the context of his experience with so-called ‘jeweled self-dribbling basketballs’ he would encounter on DMT trips, who he said were made of a kind of language, or ‘syntax binding light’: “You wonder what to make of it. I’ve thought about this for years and years and years, and I don’t know why there should be an invisible syntactical intelligence giving language lessons in hyperspace. That certainly, consistently seems to be what is happening. I’ve thought a lot about language as a result of that. First of all, it is the most remarkable thing we do. Chomsky showed the deep structure of language is under genetic control, but that’s like the assembly language level. Local expressions of language are epigenetic. It seems to me that language is some kind of enterprise of human beings that is not finished. We have now left the grunts and the digs of the elbow somewhat in the dust. But the most articulate, brilliantly pronounced and projected English or French or German or Chinese is still a poor carrier of our intent. A very limited bandwidth for the intense compression of data that we are trying to put across to each other. Intense compression. It occurs to me, the ratios of the senses, the ratio between the eye and the ear, and so forth, this also is not genetically fixed. There are ear cultures and there are eye cultures. Print cultures and electronic cultures. So, it may be that our perfection and our completion lies in the perfection and completion of the word. Again, this curious theme of the word and its effort to concretize itself. A language that you can see is far less ambiguous than a language that you hear. If I read the paragraph of Proust, then we could spend the rest of the afternoon discussing, what did he mean? But if we look at a piece of sculpture by Henry Moore, we can discuss, what did he mean, but at a certain level, there is a kind of shared bedrock that isn’t in the Proust passage. We each stop at a different level with the textual passage. With the three-dimensional object, we all sort of start from the same place and then work out our interpretations. Is it a nude, is it an animal? Is it bronze, is it wood? Is it poignant, is it comical? So forth and so on.” This post feels like the beginning of that concretization.
- eszed 2y ago
  Fascinating comment, that articulates the point of TFA better than TFA did. I've always been highly articulate, and also frustrated by the limitations of spoken language. This is a common (maybe even the dominant?) theme in 20th century theatrical writing. People like Ibsen, Chekhov, Pinter, Genet, and Churchill all struggle with it in their own ways. People like Beckett and LePage and Sarah Kane ultimately kind of abandon language altogether. Or, though poetry's not as much my field as theatre, you could go back to TS Eliot: ... Words strain, Crack, and sometimes break, under the burden, Under the tension, slip, slide, perish, Decay with imprecision, will not stay in place, Will not stay still. My own speculation, along your lines, is that it's because sound is transient, hearing imperfect, and memory fallible. Even apart from ambiguity, two people will never quite agree on what was said. (Most of my arguments with my wife begin this way!) Even court transcripts, intended to eliminate this limitation, don't capture non-verbal cues. As someone who's been marinated in the written and spoken word for all my life, research like this is fascinating, and slightly creepy: will all of the ghosts in the machine be exorcised? If those are blown away, and the bare mechanism of language exposed, what comes next?
deleted 2y ago
[deleted]
kaycebasques 2y ago
> Looking at this plot by @oca.computer, I feel like I’m peering into the world’s first microscope and spying bacteria, or through a blurry, early telescope, and spotting invisible dots that turn out to be the previously unknown moons of Jupiter… There is something there! New information to be interpreted!
- 1024core 2y ago
  Any tools to replicate @oca.computer's work? Once we have the 1000-dim vector embeddings I can make the rest work. Not sure how to go from 20-word span to a 1000-dim vector embedding.
  - 10c8 2y ago
    Generating embeddings is relatively simple with a model and Python code. There's plenty of them on HuggingFace, along with code examples. all-MiniLM-L6-v2 is a really (if not the most) popular one (albeit not SotA), with 384 dimensions: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v... Edit: A more modern and robust suite of models comes from Nomic, and can generate embeddings with 64 to 768 dimensions (https://huggingface.co/nomic-ai/nomic-embed-text-v1.5 https://huggingface.co/nomic-ai/nomic-embed-text-v1.5). When the author talks about thousands of dimensions, they're probably talking about the OpenAI embedding models.
failrate 2y ago
For a game based on semantic vectors: https://semantle.com/ https://semantle.com/
nkurz 2y ago
> "Even in 1821, horses were wrongly depicted running like dogs." Great essay, but this small comment toward the end of the essay confused me. Is he saying that dogs never gallop? I'm still not sure about the answer breed-by-breed, but searching for it led me to this interesting page illustrating different dog gaits: https://vanat.ahc.umn.edu/gaits/index.html https://vanat.ahc.umn.edu/gaits/index.html In particular, it seems to say that at least some dogs do the same "transverse gallop" that horses use: https://vanat.ahc.umn.edu/gaits/transGallop.html https://vanat.ahc.umn.edu/gaits/transGallop.html And that greyhounds at least also do a "rotary gallop": https://vanat.ahc.umn.edu/gaits/rotGallop.html https://vanat.ahc.umn.edu/gaits/rotGallop.html I have a Vizsla (one of several breeds in the running for second fastest breed after greyhounds) and my guess is that she at times does both gallops. I can't find a reference to confirm this, though.
- deleted 2y ago
  [deleted]
- Maken 2y ago
  In the linked article (https://www.amusingplanet.com/2019/06/the-galloping-horse-problem-and-worlds.html https://www.amusingplanet.com/2019/06/the-galloping-horse-pr...) there are some examples of "wrong" galloping horses. The first two examples look like the "rotary gallop", which is how a dog or a cat, not a horse, would run. The third example is plainly wrong, because the horses are mid-air but seemly ready to land in one leg.
deleted 2y ago
[deleted]
mortenjorck 2y ago
Yes, yes, more explorations in this direction. For a couple of years now, I've had this half-articulated sense that the uncanny ability of sufficiently-advanced language models to back into convincing simulations of conscious thought entirely via predicting language tokens means something profound about the nature of language itself. I'm sure there are much smarter people than I thinking about this (and probably quite a bit of background reading that would help; Chomsky, perhaps McLuhan?) but it feels like, in parallel to everything going on in the development of LLMs, there's also something big about us waiting there under the surface.
- skydhash 2y ago
  > convincing simulations of conscious thought entirely via predicting language tokens means something profound about the nature of language itself. > there's also something big about us waiting there under the surface. I don't believe so. In "The Origins of Knowledge and Imagination" by Jacob Brownoski, he argues that human language have four unique characteristics: - We can separate information (data of what being described) from emotional content (how we're supposed to react). There's no longer a bijection between communication and action. - We can extend the time reference of the communication content. We talk about the past, we plan for the future. - We can refer to ourselves. So we examine what we've done and iterate over it until we fix the errors. We can see ourselves doing the action without actually doing it. - We can rearrange units of languages to have different meanings. The same words can have different meanings based on their order. So meaning depends not only on the words, but their sequence. And that goes from words to phrases to sequence of dialogs. The fourth point is the most important. LLMs by predicting languages tokens can give use the most common order for a particular context. And because we don't have that many words, their orders can be extracted from books and other written content. But then they fail for the higher levels, mostly because that's when everything get unique. As for the third point, by observing ourselves, our communication is constantly being based on reality, which grounds it in truth. And because we can extend the reference it's based on, that leads us to observe changes and model laws. The first point allows us to separate what things are from what we should do or feel based on their existence and absence. Instead of the LLMs fooling us, it's more us fooling ourselves, because by recognizing meaning in sentences, we try to extract meanings for longer sequences of text where there aren't any. Why? Because there is no "I" that has done the job of extracting information and using language to transmit it (while still cognizant of the imperfection of natural languages). LLMs are lossy compressions of ideas. Only the smallest survives and then it generates much more false ones.
lettergram 2y ago
Quite literally what my company does - https://ipcopilot.ai/ https://ipcopilot.ai/ We discover innovative ideas in companies and help them protect their IP.
Terr_ 2y ago
> What if the difference between statements that are simply speculative and statement that mislead are as obvious as, I don’t know, the difference between a photo and a hand-drawn sketch? Given how long these have been pored over by existing hyperconnected nanomachine networks (i.e. brains) it may be that we'll mostly unearth qualities humans can already detect, even if only subconsciously. When it comes to separating truth and lies, perhaps the real trick the computer will bring is removing context, e.g. scoring text without confirmation bias towards its conclusion.
- TeMPOraL 2y ago
  LLMs seem to do more of what brains do unconsciously, rather than consciously. Which means brains may be better at rating e.g. trustworthiness of some text, but they don't surface specific ratings to the conscious level. Meanwhile, language models seem to be able to expose those features as knobs, allowing you to boost or attenuate them. So you get to drag the e.g. "excited" slider down to minimum, and get a text that may be easier to process at a conscious level. Having a slider to remove rhetoric from text would be really useful development.

249 points by RA2lover 2y ago

> What would it mean to listen to a politician speak on TV, and in real-time see a rhetorical manoeuvre that masks a persuasive bait and switch? Why do I suspect the offence will always be ahead of the defence in these areas? I'd earlier suggested that everyone, in elementary school, ought to watch Ancient Aliens and attempt to note the moment where each episode jumps the shark. I take it we could attempt this with LLMs, now?

rablackburn 2y ago
> Why do I suspect the offence will always be ahead of the defence in these areas? because destroying is easier than creating/entropy increases over time? The only solution I can see is working on turning bad actors into good actors, or another way: positive reinforcement cycles. No idea what that would look like with regard to LLMs though.
- pixl97 2y ago
  At the end of the day there is no permanent solution. In nature we typically don't see something 'win' and that's the end of the story. I mean yes things do go extinct, but the winner always has something new to deal with. Could be a more advanced predator eating all it's food sources. Could be a bacteria that it's not resistant to. Simply put, when there's entropy on the table, something is going to evolve to take it with the least amount of work possible.

dhosek 2y ago

For those perplexed by the headline, the Muybridge camera moment refers to Eadweard Muybridge who managed via camera photos taken in rapid succession to prove that when a horse runs it at times has all four legs above the ground. https://en.wikipedia.org/wiki/Eadweard_Muybridge https://en.wikipedia.org/wiki/Eadweard_Muybridge (the article doesn’t bother to mention any of this until near the end in the tl;dr section, which since it’s tl and you dr, you never got to).

stavros 2y ago
Not only that, but the tldr basically only talks about that, so it's not much of a summary at all. I read the tldr and I have no idea what the article is about.
Animats 2y ago
(On an irrelevant note, the Stanford Barn, where those pictures were taken, has gradually been closed off to the world. It was open to the public until COVID. It's still there, and there's a Stanford equestrian team, but road access has been cut and all mentions of the barn removed from directional signs.)
- gausswho 2y ago
  There are so many of these places I've encountered what used to be publicly available pre-COVID and are no longer. The reasons/excuses vary. Example: Sometimes it's a symptom of a small business already wanted a reason to pivot to a new venture, and they keep the old thing going to profit from some old whales while in transition.
  - dhosek 2y ago
    There was a lot of that post 9/11 too. It used to be that you could walk into nearly any office building in the world with little more than a smile and a confident wave. A lot of previously public areas got locked down on September 12th.
    - PopAlongKid 2y ago
      Office building security changed significantly much earlier than 2001. The mass shooting in 1993 at 101 California Street in San Francisco was the beginning of many such changes. The attack [...] also precipitated sweeping changes in downtown San Francisco. Before Ferri walked into the building that July day, almost no high-rises in the city had security measures. While many had a front desk, only a handful checked badges. The building at 101 California had two side entrances that were completely unguarded. The Examiner reported that at the time, the Chevron building and Charles Schwab’s SF headquarters had the toughest security in town; electronic badges were required at Chevron, an anomaly in 1993. Today, security checks are standard at offices large and small, a fundamental shift that happened because of 101 California. https://www.police1.com/active-shooter/articles/101-california-the-high-rise-shooting-that-changed-san-francisco-95NXjtYNeX3sBOFn/ https://www.police1.com/active-shooter/articles/101-californ...

qup 2y ago

https://archive.is/EcQfE https://archive.is/EcQfE Site is struggling

anigbrowl 2y ago

Zardoz predicted this ~50 years ago

nickreese 2y ago

I thoroughly enjoyed reading this style of loose connected thoughts.

kepano 2y ago

The repercussions of what the author summarizes as "could you colour-grade a book?" still feel wildly unknown to me, even after a couple years of thinking about it (see Photoshop for text [1][2]). Partially it's because we're still wrapping our heads around what kind of experience this might enable. The tools still feel ahead of the medium. I think we're closer to Niépce than Muybridge. In photography terms, we've just figured out how to capture photons on paper — and artists haven't figured out how to use that to make something interesting. [1] https://news.ycombinator.com/item?id=33253606 https://news.ycombinator.com/item?id=33253606 [2] https://stephango.com/photoshop-for-text https://stephango.com/photoshop-for-text

throw46365 2y ago
> The tools still feel ahead of the medium. Or maybe it's that we instinctively feel that writing should still be linear writing, if reading is still going to be linear reading. Personally I think the "photoshop for text" analogy shows just how misguided it is to expect people to tolerate words that were calculated, not crafted. Literacy is too important to mess with like this.
- kepano 2y ago
  Genuine question — do you think synthetic images pose less of a problem than synthetic text? If yes, why?
  - throw46365 2y ago
    Images — photos, paintings, designs — are not primary human expression. Words are fundamental, dense, often objectively chosen, and the most primary way of communicating thoughts. Asking someone to read your thoughts that you didn’t actually even think, because you’d rather save the time writing them, is profoundly disrespectful to the reader, who has to invest the same amount of time reading generated words as real ones. Which is not to say that I think passing off generative images as one’s own work is not disrespectful. Or that extensive, unreal body sculpting or skin retouching is not — as a photographer I believe that to also often be not just unethical but immoral. But a judgement on a retouched image is less of a burden of time. I would likely judge someone who uses ChatGPT to communicate personally with me as harshly as I would judge them editing a photo to deliberately lie to me. (Which is not to say that I don’t think GPTs have inherent grammatical advantages for cleaning up poorly-written text; I do think generating entirely new text is disrespectful to the reader, though)

Animats 2y ago

So embedding space itself is interesting. It's more than a step to an LLM. That's been known for a while, back to that early result where "King" - "Man" + "Woman" -> "Queen". This article, though, suggests more uses for embedding spaces. This could be interesting. It's a step beyond viewing them as a black box.

082349872349872 2y ago
Is ♔ - m + f = ♕ specific to embeddings, or does it also work in https://en.wikipedia.org/wiki/Formal_concept_analysis#Example https://en.wikipedia.org/wiki/Formal_concept_analysis#Exampl... ? (either as ♔ ⊕ f ⊕ m = ♕ or as ♔ ⋀ not(m) ⋁ f = ♕?) [alas, HN scrubs venus and mars symbols, and I shall spare you all the ancient egyptian hieroglyphs and O'Keeffean mathematical symbols, so `f` and `m` they are]

szvsw 2y ago

One thing I always find interesting but not discussed all that much at least in things I’ve read is - what happens in the spaces between the data? Obviously this is an incredibly high dimensional space which is only sparsely populated by the entirety of the English language; all tokens, etc. if the space is truly structured well enough, then there is a huge amount of interesting, implicit, almost platonic meaning occurring in the spaces between the data - synthetic? Dialectic? Idk. Anyways, I think those areas are a space that algorithmic intelligence will be able to develop its own notions of semantics and creativity in expression. Things that might typically be ineffable may find easy expression somewhere in embedding space. Heidegger’s thisness might be easily located somewhere in a latent representation… this is probably some linguistics 101 stuff but it’s still fascinating imo.

Der_Einzige 2y ago
Ya I'm having my return to plato moment. It really feels like we are the dēmiurgós right now with AI systems. The nature of interpolation vs extrapolation and the exploration of latent spaces will answer a lot of philosophical questions that we didn't expect to be answered so quickly, and by computers of all things.
skydhash 2y ago
I strongly believe there's nothing there other than gibberish. Piping /dev/random to a word selector will probably enumerates everything inside that set. There's a reason we can translate between every language on earth. That's because it's the same earth and reality. So there's a common sets of concepts that gives us the foundational rules of languages. Which is the data that you're speaking about.
mortenjorck 2y ago
Now this is a fun idea. If you think of embeddings as a sort of quantization of latent space, what would happen if you “turned off” that quantization? It would obviously make no sense to us, as we can only understand the output of vectors that map to tokens in languages we speak, but you could imagine a language model writing something in a sort of platonic, infinitely precise language that another model with the same latent space could then interpret.

zharknado 2y ago

> Could you dynamically change the register or tone of text depending on audience, or the reading age, or dial up the formality or subjective examples or mentions of wildlife, depending on the psychological fingerprint of the reader or listener? This seems plausible, and amazing or terrible depending on the application. An amazing application would be textbooks that adapt to use examples, analogies, pacing, etc. that enhance the reader’s engagement and understanding. An unfortunate application would be mapping which features are persuasive to individual users for hyper-targeted advertising and propaganda. A terrible application would be tracking latent political dissent to punish people for thought-crime.

lsaferite 2y ago
I'm sure it comes up frequently, but the adapting textbook thought reminds me of the "Young Lady's Illustrated Primer" from Diamond Age.

sebmellen 2y ago

Terence McKenna phrased this wonderfully, by saying “It seems to me that language is some kind of enterprise of human beings that is not finished.” The full quote is more psychedelic, in the context of his experience with so-called ‘jeweled self-dribbling basketballs’ he would encounter on DMT trips, who he said were made of a kind of language, or ‘syntax binding light’: “You wonder what to make of it. I’ve thought about this for years and years and years, and I don’t know why there should be an invisible syntactical intelligence giving language lessons in hyperspace. That certainly, consistently seems to be what is happening. I’ve thought a lot about language as a result of that. First of all, it is the most remarkable thing we do. Chomsky showed the deep structure of language is under genetic control, but that’s like the assembly language level. Local expressions of language are epigenetic. It seems to me that language is some kind of enterprise of human beings that is not finished. We have now left the grunts and the digs of the elbow somewhat in the dust. But the most articulate, brilliantly pronounced and projected English or French or German or Chinese is still a poor carrier of our intent. A very limited bandwidth for the intense compression of data that we are trying to put across to each other. Intense compression. It occurs to me, the ratios of the senses, the ratio between the eye and the ear, and so forth, this also is not genetically fixed. There are ear cultures and there are eye cultures. Print cultures and electronic cultures. So, it may be that our perfection and our completion lies in the perfection and completion of the word. Again, this curious theme of the word and its effort to concretize itself. A language that you can see is far less ambiguous than a language that you hear. If I read the paragraph of Proust, then we could spend the rest of the afternoon discussing, what did he mean? But if we look at a piece of sculpture by Henry Moore, we can discuss, what did he mean, but at a certain level, there is a kind of shared bedrock that isn’t in the Proust passage. We each stop at a different level with the textual passage. With the three-dimensional object, we all sort of start from the same place and then work out our interpretations. Is it a nude, is it an animal? Is it bronze, is it wood? Is it poignant, is it comical? So forth and so on.” This post feels like the beginning of that concretization.

eszed 2y ago
Fascinating comment, that articulates the point of TFA better than TFA did. I've always been highly articulate, and also frustrated by the limitations of spoken language. This is a common (maybe even the dominant?) theme in 20th century theatrical writing. People like Ibsen, Chekhov, Pinter, Genet, and Churchill all struggle with it in their own ways. People like Beckett and LePage and Sarah Kane ultimately kind of abandon language altogether. Or, though poetry's not as much my field as theatre, you could go back to TS Eliot: ... Words strain, Crack, and sometimes break, under the burden, Under the tension, slip, slide, perish, Decay with imprecision, will not stay in place, Will not stay still. My own speculation, along your lines, is that it's because sound is transient, hearing imperfect, and memory fallible. Even apart from ambiguity, two people will never quite agree on what was said. (Most of my arguments with my wife begin this way!) Even court transcripts, intended to eliminate this limitation, don't capture non-verbal cues. As someone who's been marinated in the written and spoken word for all my life, research like this is fascinating, and slightly creepy: will all of the ghosts in the machine be exorcised? If those are blown away, and the bare mechanism of language exposed, what comes next?

deleted 2y ago

[deleted]

kaycebasques 2y ago

> Looking at this plot by @oca.computer, I feel like I’m peering into the world’s first microscope and spying bacteria, or through a blurry, early telescope, and spotting invisible dots that turn out to be the previously unknown moons of Jupiter… There is something there! New information to be interpreted!

1024core 2y ago
Any tools to replicate @oca.computer's work? Once we have the 1000-dim vector embeddings I can make the rest work. Not sure how to go from 20-word span to a 1000-dim vector embedding.
- 10c8 2y ago
  Generating embeddings is relatively simple with a model and Python code. There's plenty of them on HuggingFace, along with code examples. all-MiniLM-L6-v2 is a really (if not the most) popular one (albeit not SotA), with 384 dimensions: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 https://huggingface.co/sentence-transformers/all-MiniLM-L6-v... Edit: A more modern and robust suite of models comes from Nomic, and can generate embeddings with 64 to 768 dimensions (https://huggingface.co/nomic-ai/nomic-embed-text-v1.5 https://huggingface.co/nomic-ai/nomic-embed-text-v1.5). When the author talks about thousands of dimensions, they're probably talking about the OpenAI embedding models.

failrate 2y ago

For a game based on semantic vectors: https://semantle.com/ https://semantle.com/

nkurz 2y ago

> "Even in 1821, horses were wrongly depicted running like dogs." Great essay, but this small comment toward the end of the essay confused me. Is he saying that dogs never gallop? I'm still not sure about the answer breed-by-breed, but searching for it led me to this interesting page illustrating different dog gaits: https://vanat.ahc.umn.edu/gaits/index.html https://vanat.ahc.umn.edu/gaits/index.html In particular, it seems to say that at least some dogs do the same "transverse gallop" that horses use: https://vanat.ahc.umn.edu/gaits/transGallop.html https://vanat.ahc.umn.edu/gaits/transGallop.html And that greyhounds at least also do a "rotary gallop": https://vanat.ahc.umn.edu/gaits/rotGallop.html https://vanat.ahc.umn.edu/gaits/rotGallop.html I have a Vizsla (one of several breeds in the running for second fastest breed after greyhounds) and my guess is that she at times does both gallops. I can't find a reference to confirm this, though.

deleted 2y ago
[deleted]
Maken 2y ago
In the linked article (https://www.amusingplanet.com/2019/06/the-galloping-horse-problem-and-worlds.html https://www.amusingplanet.com/2019/06/the-galloping-horse-pr...) there are some examples of "wrong" galloping horses. The first two examples look like the "rotary gallop", which is how a dog or a cat, not a horse, would run. The third example is plainly wrong, because the horses are mid-air but seemly ready to land in one leg.

deleted 2y ago

[deleted]

mortenjorck 2y ago

Yes, yes, more explorations in this direction. For a couple of years now, I've had this half-articulated sense that the uncanny ability of sufficiently-advanced language models to back into convincing simulations of conscious thought entirely via predicting language tokens means something profound about the nature of language itself. I'm sure there are much smarter people than I thinking about this (and probably quite a bit of background reading that would help; Chomsky, perhaps McLuhan?) but it feels like, in parallel to everything going on in the development of LLMs, there's also something big about us waiting there under the surface.

skydhash 2y ago
> convincing simulations of conscious thought entirely via predicting language tokens means something profound about the nature of language itself. > there's also something big about us waiting there under the surface. I don't believe so. In "The Origins of Knowledge and Imagination" by Jacob Brownoski, he argues that human language have four unique characteristics: - We can separate information (data of what being described) from emotional content (how we're supposed to react). There's no longer a bijection between communication and action. - We can extend the time reference of the communication content. We talk about the past, we plan for the future. - We can refer to ourselves. So we examine what we've done and iterate over it until we fix the errors. We can see ourselves doing the action without actually doing it. - We can rearrange units of languages to have different meanings. The same words can have different meanings based on their order. So meaning depends not only on the words, but their sequence. And that goes from words to phrases to sequence of dialogs. The fourth point is the most important. LLMs by predicting languages tokens can give use the most common order for a particular context. And because we don't have that many words, their orders can be extracted from books and other written content. But then they fail for the higher levels, mostly because that's when everything get unique. As for the third point, by observing ourselves, our communication is constantly being based on reality, which grounds it in truth. And because we can extend the reference it's based on, that leads us to observe changes and model laws. The first point allows us to separate what things are from what we should do or feel based on their existence and absence. Instead of the LLMs fooling us, it's more us fooling ourselves, because by recognizing meaning in sentences, we try to extract meanings for longer sequences of text where there aren't any. Why? Because there is no "I" that has done the job of extracting information and using language to transmit it (while still cognizant of the imperfection of natural languages). LLMs are lossy compressions of ideas. Only the smallest survives and then it generates much more false ones.

lettergram 2y ago

Quite literally what my company does - https://ipcopilot.ai/ https://ipcopilot.ai/ We discover innovative ideas in companies and help them protect their IP.

Terr_ 2y ago

> What if the difference between statements that are simply speculative and statement that mislead are as obvious as, I don’t know, the difference between a photo and a hand-drawn sketch? Given how long these have been pored over by existing hyperconnected nanomachine networks (i.e. brains) it may be that we'll mostly unearth qualities humans can already detect, even if only subconsciously. When it comes to separating truth and lies, perhaps the real trick the computer will bring is removing context, e.g. scoring text without confirmation bias towards its conclusion.

TeMPOraL 2y ago
LLMs seem to do more of what brains do unconsciously, rather than consciously. Which means brains may be better at rating e.g. trustworthiness of some text, but they don't surface specific ratings to the conscious level. Meanwhile, language models seem to be able to expose those features as knobs, allowing you to boost or attenuate them. So you get to drag the e.g. "excited" slider down to minimum, and get a text that may be easier to process at a conscious level. Having a slider to remove rhetoric from text would be really useful development.