8 ms·

Pride, Prejudice

95 points by enkiv2 9y ago

sincerely 9y ago
This is obviously just for fun, but it would be interesting to see if a similar approach could be considered for, say, high-school level textbooks, which are notoriously padded with useless content.
em3rgent0rdr 9y ago
When will the browser plugin come out? :)
- Xeoncross 9y ago
  Here's what I have so far: var p=document.createElement('pre'); p.textContent=document.body.textContent.replace(/[ \t]+/ig, ' ').replace(/\n[\s\n]*/ig, "\n\n"); document.body.replaceWith(p);
schoen 9y ago
Interestingly, American and British English disagree about the contraction of possessive "have". In American English, "have" and "had" can be contracted when they are used as an auxiliary (to have done something) but not when used as an possessive (to have something). I've seen that before. I'd seen that before. But not: ∗I've three bicycles. ∗He'd problems with spelling in elementary school. However, British English does allow this contraction! The text creates a contraction of "have had" to "have'd", where the first is an auxiliary and the second is a possessive (following an adverb rather than the subject, so that the auxiliary can't contract), which I think is proscribed in both varieties of English: ?I certainly have'd my share of beauty But I'm not positive about the British assessment of this.
- jszymborski 9y ago
  I'm Canadian, not from the UK, but I can't imagine contracting possessive nouns with had. Then again, this software contracts the with nouns rather aggressively ("this truth is so well fixed in t'minds of t'surrounding families") which is something I've never heard of either.
  - gerdesj 9y ago
    To be honest I think that would probably work best as: "this truth is so well fixed in t'minds of surrounding families" Simply drop the second definite article because it is redundant. Actually you could drop the first one as well. I can hear it in a generic North England accent and "in t'minds" would actually sound more like "int minds" but trying to run the word "of" into "t'" is a bit harder so it would probably be dropped entirely. I speak RP English with a touch of Mummerset ...
- gerdesj 9y ago
  Yes all of those are "allowed" in en_GB. However, the second example you give would almost certainly be rendered as "He'd had problems ...". Yes, really! Thinking about it, it depends, for your first example: "Who on earth has three bicycles?" Tom asked, incredulously. "Well, I've got three", replied Jim. "I've three bicycles up for sale today" or "There're three bikes ..." or "There are three bikes". Some people find there're (there are) a bit dodgy. I don't think that en_GB or any other version really gets too hung up on things like that anymore. That said, some constructions do seem more natural than others. I have never heard or seen anyone try to use "have'd" in any modern form of English. You can get "I've had" from "I have had" or "I'd 've had" from "I would have had" or if you really want to drop the aitches then I'd 've 'ad. The last two are quite common in speech but not written.
  - cholantesh 9y ago
    What about "He'd'd problems with spelling..."? Theoretically it seems fine.
jamesjyu 9y ago
I applaud trying out cool techniques to transform text, but I think this would work better for non-fiction books. Fiction is like a serialization of senses for your inner director that paints a world in your mind. By lopping off words, you are altering the pacing and tone of the book. Now, maybe your intent is to create a book that has a totally different character than the original, and if it is, then that's fine. But if your intent is to maintain the character of the book, doing a few replacements can do a lot of harm. Also, most people want to read longer fiction work and be immersed for a while. This is the reason it's generally harder to sell novellas. PS. Using the 't' contraction instead of 'the' makes this really hard to parse.
- phillc73 9y ago
  > PS. Using the 't' contraction instead of 'the' makes this really hard to parse. Only in some cases. "...by a young man of large fortune from t'north of England;"[1] This is just about the perfect edit. [1] https://github.com/hugovk/NaNoGenMo-2017/blob/master/03-reducifier/output.txt#L35 https://github.com/hugovk/NaNoGenMo-2017/blob/master/03-redu...
  - deckar01 9y ago
    I have never seen "the" abbreviated (or contracted?) into "t'" before. Is this common in some dialects of English?
    - phillc73 9y ago
      Not a great reference, but best I can find on short notice: https://www.wikihow.com/Speak-With-a-Yorkshire-Accent https://www.wikihow.com/Speak-With-a-Yorkshire-Accent Part 1, Section 4. Edit: The Vocabulary and Grammar section here discusses Definite Article Reduction: https://en.wikipedia.org/wiki/Yorkshire_dialect#Vocabulary_and_grammar https://en.wikipedia.org/wiki/Yorkshire_dialect#Vocabulary_a...
    - gerdesj 9y ago
      Yes it is. It is common in Yorkshire and Derbyshire at the least - so roughly the north Midlands and up in England. Probably several other regions use it as well. The North of England is often referred to as t'North by pretty much everyone. It is sometimes pronounced a bit like "tut" with the second t silent. The following word is often affected as well. Actually its bloody hard to try and describe. Your best bet is to find something on Youtube spoken with a Yorkshire accent. Good luck trying to understand it!
leafyape 9y ago
Readability suffers and character reduction will be nowhere near the rate of word reduction. But fun idea.
- twic 9y ago
  Character reduction? Well, i suppose Pride and Prejudice could probably stand to lose two or three Bennet sisters.
  - cholantesh 9y ago
    It absolutely cannot, sir.
supermdguy 9y ago
Great to see a NaNoGenMo entry on HN! If you've never participated, it's a competition (with no official winners) to see who can write a program that generates a 50k word "novel" in one month. It happens each November, to parallel NaNoWriMo. See https://nanogenmo.github.io/ https://nanogenmo.github.io/ for more details and other entries.
ihaveajob 9y ago
For added fun and compression, choose American spelling of words like "honour" and "neighbour".
voidhorse 9y ago
This is obviously a cheeky/fun project, but I'll just take a moment to be pedantic and point out that literature is entirely the wrong domain for this sort of tool. A litany of literary information is encoded in the superfluous: authorial idiosyncrasies, period styles, the intended regional locale of the characters, the anticipated reading level of the audience/familiarity with the material, meaning at the linguistic level ('meta' narrative). While I think this tool has a great deal of potential application in domains like technical documents, texting (or other forms of communication that lend themselves to brevity) etc. I don't think literature falls into that camp.
QML 9y ago
Hmm, a comment from someone who just learned about Huffman encoding and had a project on text classification -- it seems that the majority of commonly-written or spoken English words are terrible for conveying semantics; words like "the", "a", "to" are mostly just filler words. Is it possible then to create a vocabulary of English words dedicated towards an emphasis on meaning?
gt_ 9y ago
Congrats on the project! It's hilarious, probably useful somewhere, and it made my day. Something in the opposite direction would be interesting as well. As a student, I was selective with where I invested my energy, according to my level of interest in the topic. Like many students, I developed an obnoxious flair for elaboration where it was no longer necessarily required and, as I thought, I actually became quite good at making sentences twice their original word length and sentence length from what they were otherwise required by my instructors to be. I would have welcomed a more automated process for the task.
nautilus12 9y ago
Not being a downer, but really, is there any usefulness to this at all?
- em3rgent0rdr 9y ago
  Presumably can read the book faster because there are fewer words, while still basically retaining the same amount of information.
  - nautilus12 9y ago
    Id venture to say this almost completely contrary to the point of literature. Literature is not about the information contained within, its about how the information is presented.

95 points by enkiv2 9y ago

This is obviously just for fun, but it would be interesting to see if a similar approach could be considered for, say, high-school level textbooks, which are notoriously padded with useless content.

em3rgent0rdr 9y ago

When will the browser plugin come out? :)

Xeoncross 9y ago
Here's what I have so far: var p=document.createElement('pre'); p.textContent=document.body.textContent.replace(/[ \t]+/ig, ' ').replace(/\n[\s\n]*/ig, "\n\n"); document.body.replaceWith(p);

schoen 9y ago

Interestingly, American and British English disagree about the contraction of possessive "have". In American English, "have" and "had" can be contracted when they are used as an auxiliary (to have done something) but not when used as an possessive (to have something). I've seen that before. I'd seen that before. But not: ∗I've three bicycles. ∗He'd problems with spelling in elementary school. However, British English does allow this contraction! The text creates a contraction of "have had" to "have'd", where the first is an auxiliary and the second is a possessive (following an adverb rather than the subject, so that the auxiliary can't contract), which I think is proscribed in both varieties of English: ?I certainly have'd my share of beauty But I'm not positive about the British assessment of this.

jszymborski 9y ago
I'm Canadian, not from the UK, but I can't imagine contracting possessive nouns with had. Then again, this software contracts the with nouns rather aggressively ("this truth is so well fixed in t'minds of t'surrounding families") which is something I've never heard of either.
- gerdesj 9y ago
  To be honest I think that would probably work best as: "this truth is so well fixed in t'minds of surrounding families" Simply drop the second definite article because it is redundant. Actually you could drop the first one as well. I can hear it in a generic North England accent and "in t'minds" would actually sound more like "int minds" but trying to run the word "of" into "t'" is a bit harder so it would probably be dropped entirely. I speak RP English with a touch of Mummerset ...
gerdesj 9y ago
Yes all of those are "allowed" in en_GB. However, the second example you give would almost certainly be rendered as "He'd had problems ...". Yes, really! Thinking about it, it depends, for your first example: "Who on earth has three bicycles?" Tom asked, incredulously. "Well, I've got three", replied Jim. "I've three bicycles up for sale today" or "There're three bikes ..." or "There are three bikes". Some people find there're (there are) a bit dodgy. I don't think that en_GB or any other version really gets too hung up on things like that anymore. That said, some constructions do seem more natural than others. I have never heard or seen anyone try to use "have'd" in any modern form of English. You can get "I've had" from "I have had" or "I'd 've had" from "I would have had" or if you really want to drop the aitches then I'd 've 'ad. The last two are quite common in speech but not written.
- cholantesh 9y ago
  What about "He'd'd problems with spelling..."? Theoretically it seems fine.

jamesjyu 9y ago

I applaud trying out cool techniques to transform text, but I think this would work better for non-fiction books. Fiction is like a serialization of senses for your inner director that paints a world in your mind. By lopping off words, you are altering the pacing and tone of the book. Now, maybe your intent is to create a book that has a totally different character than the original, and if it is, then that's fine. But if your intent is to maintain the character of the book, doing a few replacements can do a lot of harm. Also, most people want to read longer fiction work and be immersed for a while. This is the reason it's generally harder to sell novellas. PS. Using the 't' contraction instead of 'the' makes this really hard to parse.

phillc73 9y ago
> PS. Using the 't' contraction instead of 'the' makes this really hard to parse. Only in some cases. "...by a young man of large fortune from t'north of England;"[1] This is just about the perfect edit. [1] https://github.com/hugovk/NaNoGenMo-2017/blob/master/03-reducifier/output.txt#L35 https://github.com/hugovk/NaNoGenMo-2017/blob/master/03-redu...
- deckar01 9y ago
  I have never seen "the" abbreviated (or contracted?) into "t'" before. Is this common in some dialects of English?
  - phillc73 9y ago
    Not a great reference, but best I can find on short notice: https://www.wikihow.com/Speak-With-a-Yorkshire-Accent https://www.wikihow.com/Speak-With-a-Yorkshire-Accent Part 1, Section 4. Edit: The Vocabulary and Grammar section here discusses Definite Article Reduction: https://en.wikipedia.org/wiki/Yorkshire_dialect#Vocabulary_and_grammar https://en.wikipedia.org/wiki/Yorkshire_dialect#Vocabulary_a...
  - gerdesj 9y ago
    Yes it is. It is common in Yorkshire and Derbyshire at the least - so roughly the north Midlands and up in England. Probably several other regions use it as well. The North of England is often referred to as t'North by pretty much everyone. It is sometimes pronounced a bit like "tut" with the second t silent. The following word is often affected as well. Actually its bloody hard to try and describe. Your best bet is to find something on Youtube spoken with a Yorkshire accent. Good luck trying to understand it!

leafyape 9y ago

Readability suffers and character reduction will be nowhere near the rate of word reduction. But fun idea.

twic 9y ago
Character reduction? Well, i suppose Pride and Prejudice could probably stand to lose two or three Bennet sisters.
- cholantesh 9y ago
  It absolutely cannot, sir.

supermdguy 9y ago

Great to see a NaNoGenMo entry on HN! If you've never participated, it's a competition (with no official winners) to see who can write a program that generates a 50k word "novel" in one month. It happens each November, to parallel NaNoWriMo. See https://nanogenmo.github.io/ https://nanogenmo.github.io/ for more details and other entries.

ihaveajob 9y ago

For added fun and compression, choose American spelling of words like "honour" and "neighbour".

voidhorse 9y ago

This is obviously a cheeky/fun project, but I'll just take a moment to be pedantic and point out that literature is entirely the wrong domain for this sort of tool. A litany of literary information is encoded in the superfluous: authorial idiosyncrasies, period styles, the intended regional locale of the characters, the anticipated reading level of the audience/familiarity with the material, meaning at the linguistic level ('meta' narrative). While I think this tool has a great deal of potential application in domains like technical documents, texting (or other forms of communication that lend themselves to brevity) etc. I don't think literature falls into that camp.

QML 9y ago

Hmm, a comment from someone who just learned about Huffman encoding and had a project on text classification -- it seems that the majority of commonly-written or spoken English words are terrible for conveying semantics; words like "the", "a", "to" are mostly just filler words. Is it possible then to create a vocabulary of English words dedicated towards an emphasis on meaning?

gt_ 9y ago

Congrats on the project! It's hilarious, probably useful somewhere, and it made my day. Something in the opposite direction would be interesting as well. As a student, I was selective with where I invested my energy, according to my level of interest in the topic. Like many students, I developed an obnoxious flair for elaboration where it was no longer necessarily required and, as I thought, I actually became quite good at making sentences twice their original word length and sentence length from what they were otherwise required by my instructors to be. I would have welcomed a more automated process for the task.

nautilus12 9y ago

Not being a downer, but really, is there any usefulness to this at all?

em3rgent0rdr 9y ago
Presumably can read the book faster because there are fewer words, while still basically retaining the same amount of information.
- nautilus12 9y ago
  Id venture to say this almost completely contrary to the point of literature. Literature is not about the information contained within, its about how the information is presented.