5 ms·

I wonder if someone has managed to extract the quote database yet for DIYers. The device's firmware is easily available via their website - the Mac Installer c

by crote 2y ago

I wonder if someone has managed to extract the quote database yet for DIYers. The device's firmware is easily available via their website - the Mac Installer contains a pair of 9MB binaries (in addition to 200MB of Electron), which seems to just be an unencrypted ESP32 blob. Running `strings` on the blob gives plenty of human-readable stuff (including entire html pages), but nothing resembling quotes. Maybe they are compressed in some way? Assuming 140 characters per quote, the entire 13000-entry quote database should fit uncompressed in about 2MB, so who knows.

__jonas 2y ago
Here is the quotes database of one of the older projects this is (potentially) based on: https://github.com/JohannesNE/literature-clock/blob/master/litclock_annotated.csv https://github.com/JohannesNE/literature-clock/blob/master/l...
- gwern 2y ago
  13,000 seems surprisingly small, all things considered. Seems like the sort of task where you could automate it with a small LLM instance and process millions of books easily. Then extract all of the key strings and have a set of tens of thousands of ways to describe time which can be searched for as fixed-string matches, and might be of interest in its own right. (How do mentioned times change over history? Do they get more precise? Do books take place at wider ranges of the day?)
  - bigiain 2y ago
    There's only 1440 minutes in a day. So there's almost 10 quotes for each possible minute to display (assuming an even enough distribution). Even if you check the time a lot - I guess I probably check the time 3 or 4 times every day between 9:20 and my regular 9:30am meeting - it'll probably be ~15 days before I hot a duplicate quote. That feels like "enough" to me?
    - gwern 2y ago
      I'm not sure your math there on the birthday paradox is right (you only check the time during a single 10 minute range every day, no matter what? and you wouldn't look at it out of curiosity or for the pleasure of seeing a new quote?), but 15 days to the first collision doesn't seem like that long to me, for a piece of furniture that one would expect to be good for years to come and which costs $200+. And note that the number will go up rapidly from whenever the first expected collision is: only 10 quotes per minute means you're going to see a lot of duplicates after a few hundred days.
NKosmatos 2y ago
What about copyright and/or intellectual property of these quotes? Do quotes fall under fair usage law?
- swiftcoder 2y ago
  At a glance most of the quotes seem to be pre-1920's, so those should all be in the public domain. One would have to make a full audit to be entirely sure that no more recent literature has crept in.
  - 1234srose 2y ago
    For this particular product, it doesn't matter what the publication year of the book is. 1) The quotes are short and 2) the use of them is transformative, so this falls squarely within the doctrine of Fair Use. That said, I'm sure if an author or publisher of a book that's still within copyright contacted Author Clock and asked them to remove their quotes from the database they probably would, because it just wouldn't be a thing worth fighting about.
- joe5150 2y ago
  Reproducing a quotation from a book is likely always fair use in the US. A database of quotes is also probably not copyrightable in the first place though of course the software is.

by crote 2y ago

__jonas 2y ago

Here is the quotes database of one of the older projects this is (potentially) based on: https://github.com/JohannesNE/literature-clock/blob/master/litclock_annotated.csv https://github.com/JohannesNE/literature-clock/blob/master/l...

gwern 2y ago
13,000 seems surprisingly small, all things considered. Seems like the sort of task where you could automate it with a small LLM instance and process millions of books easily. Then extract all of the key strings and have a set of tens of thousands of ways to describe time which can be searched for as fixed-string matches, and might be of interest in its own right. (How do mentioned times change over history? Do they get more precise? Do books take place at wider ranges of the day?)
- bigiain 2y ago
  There's only 1440 minutes in a day. So there's almost 10 quotes for each possible minute to display (assuming an even enough distribution). Even if you check the time a lot - I guess I probably check the time 3 or 4 times every day between 9:20 and my regular 9:30am meeting - it'll probably be ~15 days before I hot a duplicate quote. That feels like "enough" to me?
  - gwern 2y ago
    I'm not sure your math there on the birthday paradox is right (you only check the time during a single 10 minute range every day, no matter what? and you wouldn't look at it out of curiosity or for the pleasure of seeing a new quote?), but 15 days to the first collision doesn't seem like that long to me, for a piece of furniture that one would expect to be good for years to come and which costs $200+. And note that the number will go up rapidly from whenever the first expected collision is: only 10 quotes per minute means you're going to see a lot of duplicates after a few hundred days.

NKosmatos 2y ago

What about copyright and/or intellectual property of these quotes? Do quotes fall under fair usage law?

swiftcoder 2y ago
At a glance most of the quotes seem to be pre-1920's, so those should all be in the public domain. One would have to make a full audit to be entirely sure that no more recent literature has crept in.
- 1234srose 2y ago
  For this particular product, it doesn't matter what the publication year of the book is. 1) The quotes are short and 2) the use of them is transformative, so this falls squarely within the doctrine of Fair Use. That said, I'm sure if an author or publisher of a book that's still within copyright contacted Author Clock and asked them to remove their quotes from the database they probably would, because it just wouldn't be a thing worth fighting about.
joe5150 2y ago
Reproducing a quotation from a book is likely always fair use in the US. A database of quotes is also probably not copyrightable in the first place though of course the software is.