4 ms·

It would be interesting to chat with an LLM that has been trained on, with increasing weight 1) all the things up to Darwin's death that he might have read, 2)

It would be interesting to chat with an LLM that has been trained on, with increasing weight 1) all the things up to Darwin's death that he might have read, 2) his own library, 3) his own words. It wouldn't sum up to Darwin's views or voice, but it would let us query them in the context in which they were written. Datomic has a feature that lets you query a database from any given point in time. If an LLM could draw its embeddings from such a database I suppose you could just ask it to respond as of 1882. I'd love to be able to repeat a query once per decade to see how our zeitgeist evolves, like "how do species change?"

xyzzy_plugh 3y ago
> query them in the context in which they were written. > I'd love to be able to repeat a query once per decade to see how our zeitgeist evolves, like "how do species change?" I think the latter is a good point re: our zeitgeist, but the former (context) is unobtainable. I struggle to imagine what that would actually represent. It's certainly not a complete representation of whatever Darwin's context was. For one, we don't read books all at once. The order in which we read them impacts our interpretation of subsequent reads. We might re-read, we might stop reading, etc. This brings me to my next point. It's not uncommon amongst collectors of books to not have read every book in their collection. A lot of double negatives there. Folks with lots of books often don't read them all. I have a small library and I have only read half of the books I own, roughly. It's not unusual for me to pickup up a book that challenges my world view, and I might not agree with it. I would not like someone else to consider its presence in my shelf as an endorsement or representation of my own views!!! I don't know much about Darwin's personal life but I imagine he didn't read all those books. Do they accurately capture the context in which his own words were written? I am doubtful. All that aside, Darwin spent considerable time abroad and at sea. He recorded much, but you can't record everything. The context of his surroundings is incredibly lossy. Even the words he did put to be paper are contextually filtered through his worldview which, let's be honest, was often focused on the next delicious animal to consume. Darwin is decidedly a great contributor to our species, but to be honest I have no real interest in talking with him, let alone a questionable facsimile thereof.
- jaimie 3y ago
  See my comment on the parent for a study I did on these questions regarding reading strategies. Of particular relevance, Darwin kept detailed records of the books he read and wished to read in two notebooks [CUL-DAR119, CUL-DAR118], which spanned from his return aboard the Beagle until a few months after The Origin of Species was published. http://darwin-online.org.uk/EditorialIntroductions/vanWyhe_notebooks.html http://darwin-online.org.uk/EditorialIntroductions/vanWyhe_n...
wara23arish 3y ago
question abt datomic isnt that just querying and filtering on timestamp?
- delichon 3y ago
  Datomic greatly simplifies it. You just give it a timestamp when you get the db handle to query on, and it returns a virtual copy of the db at that moment. They leverage the time management to make that an immutable database that you can only add to. So you have a full trace of everything ever added, and easy time travel within it, on the house. But I don't think the embeddings themselves can work like that. You'd have to train them on the corpus at each point in time. So it wouldn't be practical to query on fine time intervals because of the large amount of data or training time. It would be nice to be wrong about this.
jaimie 3y ago
This idea is very, very closely related to what I did for my PhD dissertation, with our analysis of Darwin's Readings being published in Cognition and available at https://arxiv.org/abs/1509.07175 https://arxiv.org/abs/1509.07175. Summary: Darwin kept a series of reading notebooks recording everything he read for a very convenient time-span: from his return aboard the Beagle until a few months after The Origin of Species was published. (CUL-DAR119, CUL-DAR128, http://darwin-online.org.uk/EditorialIntroductions/vanWyhe_notebooks.html http://darwin-online.org.uk/EditorialIntroductions/vanWyhe_n...) I used the bibliographic entries to look up 97% of the English, non-fiction works in his library and trained an LDA topic model over it. We observed clear behavioral shifts in his reading patterns over time, which corresponded with other biographical events. We tied this into a theory of "information foraging" - that as people are researching they are either exploiting existing topics or exploring new areas. For example, early in the reading history, one reading largely followed the next - there were few jumps in subjects between readings. As he was writing the Origin and synthesizing resource, the topic shifts became much more exploratory. This original model had no goal-directed behavior, it was only a description of what he read, but still found these behavioral shifts when he marked he was beginning the Origin. In a later study, we began to look at the "zeitgeist" question you mention and how Darwin's drafts of the Origin diverged from the culture, as represented by the books he was reading (https://arxiv.org/abs/1802.09944 https://arxiv.org/abs/1802.09944) In this second study, we used the reading model and sampled his writings into that model space. Both this library and his "to be read" notebooks define a nice "adjacent possible" - could he have read other things that may have been closer to his thoughts in the Origin? We use permutations of what he DID read as a null for studying his reading behavior - could he have read them in a more "optimal" manner? (With a lot of qualification on different "optimal" learning strategies...) Final note: the time scales of his record keeping were not rare for the time - many people of letters kept "commonplace books" that journaled these reading histories. There was once a project to collect these reading histories, but I don't have the link at hand now.
- xyzzy_plugh 3y ago
  This is wild.

xyzzy_plugh 3y ago

> query them in the context in which they were written. > I'd love to be able to repeat a query once per decade to see how our zeitgeist evolves, like "how do species change?" I think the latter is a good point re: our zeitgeist, but the former (context) is unobtainable. I struggle to imagine what that would actually represent. It's certainly not a complete representation of whatever Darwin's context was. For one, we don't read books all at once. The order in which we read them impacts our interpretation of subsequent reads. We might re-read, we might stop reading, etc. This brings me to my next point. It's not uncommon amongst collectors of books to not have read every book in their collection. A lot of double negatives there. Folks with lots of books often don't read them all. I have a small library and I have only read half of the books I own, roughly. It's not unusual for me to pickup up a book that challenges my world view, and I might not agree with it. I would not like someone else to consider its presence in my shelf as an endorsement or representation of my own views!!! I don't know much about Darwin's personal life but I imagine he didn't read all those books. Do they accurately capture the context in which his own words were written? I am doubtful. All that aside, Darwin spent considerable time abroad and at sea. He recorded much, but you can't record everything. The context of his surroundings is incredibly lossy. Even the words he did put to be paper are contextually filtered through his worldview which, let's be honest, was often focused on the next delicious animal to consume. Darwin is decidedly a great contributor to our species, but to be honest I have no real interest in talking with him, let alone a questionable facsimile thereof.

jaimie 3y ago
See my comment on the parent for a study I did on these questions regarding reading strategies. Of particular relevance, Darwin kept detailed records of the books he read and wished to read in two notebooks [CUL-DAR119, CUL-DAR118], which spanned from his return aboard the Beagle until a few months after The Origin of Species was published. http://darwin-online.org.uk/EditorialIntroductions/vanWyhe_notebooks.html http://darwin-online.org.uk/EditorialIntroductions/vanWyhe_n...

wara23arish 3y ago

question abt datomic isnt that just querying and filtering on timestamp?

delichon 3y ago
Datomic greatly simplifies it. You just give it a timestamp when you get the db handle to query on, and it returns a virtual copy of the db at that moment. They leverage the time management to make that an immutable database that you can only add to. So you have a full trace of everything ever added, and easy time travel within it, on the house. But I don't think the embeddings themselves can work like that. You'd have to train them on the corpus at each point in time. So it wouldn't be practical to query on fine time intervals because of the large amount of data or training time. It would be nice to be wrong about this.

jaimie 3y ago

This idea is very, very closely related to what I did for my PhD dissertation, with our analysis of Darwin's Readings being published in Cognition and available at https://arxiv.org/abs/1509.07175 https://arxiv.org/abs/1509.07175. Summary: Darwin kept a series of reading notebooks recording everything he read for a very convenient time-span: from his return aboard the Beagle until a few months after The Origin of Species was published. (CUL-DAR119, CUL-DAR128, http://darwin-online.org.uk/EditorialIntroductions/vanWyhe_notebooks.html http://darwin-online.org.uk/EditorialIntroductions/vanWyhe_n...) I used the bibliographic entries to look up 97% of the English, non-fiction works in his library and trained an LDA topic model over it. We observed clear behavioral shifts in his reading patterns over time, which corresponded with other biographical events. We tied this into a theory of "information foraging" - that as people are researching they are either exploiting existing topics or exploring new areas. For example, early in the reading history, one reading largely followed the next - there were few jumps in subjects between readings. As he was writing the Origin and synthesizing resource, the topic shifts became much more exploratory. This original model had no goal-directed behavior, it was only a description of what he read, but still found these behavioral shifts when he marked he was beginning the Origin. In a later study, we began to look at the "zeitgeist" question you mention and how Darwin's drafts of the Origin diverged from the culture, as represented by the books he was reading (https://arxiv.org/abs/1802.09944 https://arxiv.org/abs/1802.09944) In this second study, we used the reading model and sampled his writings into that model space. Both this library and his "to be read" notebooks define a nice "adjacent possible" - could he have read other things that may have been closer to his thoughts in the Origin? We use permutations of what he DID read as a null for studying his reading behavior - could he have read them in a more "optimal" manner? (With a lot of qualification on different "optimal" learning strategies...) Final note: the time scales of his record keeping were not rare for the time - many people of letters kept "commonplace books" that journaled these reading histories. There was once a project to collect these reading histories, but I don't have the link at hand now.

xyzzy_plugh 3y ago
This is wild.