10 ms·
Show HN: 100K sentences mined from Wikipedia to help non-native English learners
- bauer 9y agoNicely done. You could add in a mailing list to send users a digest of new or top vocabulary words every week.
- galen211 9y agoSecond that - would also be interesting for English speakers learning a language like Chinese, where the average literate speaker can recognize about 5000 of the most frequently used characters.
- peterburkimsher 9y agogalen - I'm making http://pingtype.github.io http://pingtype.github.io for learning Chinese! I don't think that studying characters is the most important thing though. Words are more important than individual characters! My program can help you type parts of a character, and put spaces between words automatically.
- steelbird 9y agoThere's a subscription link at the bottom of https://buildmyvocab.com/ https://buildmyvocab.com/
- wodenokoto 9y agoHow do you decide on which sentences to use? I'm interested in generating example sentences myself, but in a way, that chooses sentences that are simple, easy to understand and support the word, they are supposed to exemplify. For example "She got a car for her birthday, while she was traveling in Italy eating pizza" does not tell the reader anything about what a car is, or how the word should be used. However "He drives his car to work", is a much better example of what a car is, what is a common associated verb and how it fits in a sentence. How do you optimise selection for sentence like the latter?
- abhas9 9y agoThat's a great question. Optimizing for sentence selection is important for teaching. For now, I have a simple check that filters out sentences which are longer than 160 characters. Also, I believe that this is one thing which humans can do better. I, therefore, plan to add upvote & downvote buttons to rate the quality of sentences.
- nmstoker 9y agoUp/down voting seems good. I wonder if you might get a bit of an head start if you combine the shorter sentence idea with selection based on higher n-gram counts. For instance, if the keyword + words either side match a common n-gram, you could expect that sentence was reasonably representative and boost it in the initial rankings as compared to an n-gram that has a much lower count.
- deleted 9y ago[deleted]
- anchpop 9y agoI know! You could check to see if there are some verbs that are used predominately with the word you're trying to generate sentences for. Example, I would expect "drive" appearing in a sentence to carry a higher than average probability for "car" also appearing in the sentence. Or "wind" for "watch", or "sit" for "chair" and "couch". Then, I think sentences containing "car" that also contain the verb "drive" would probably give better clues for the meaning of "car" than verbs like "bought". Just a thought.
- sametmax 9y agoThat's the geek inside you talking. But the hard part was the word stats. Now that you get 1K words, writing a thousand sentences manually to illustrate them is not really hard. It's a one day manual work. Less than the work needing to figure out automation, with way better results.
- deleted 9y ago[deleted]
- cschmidt 9y agoI went to an interesting talk once at the Boston Python meetup, where a guy figured out how to order sentences so could learn them in an order where you already knew the "other" words in the sentence. Basically, making a directed graph of vocabulary. He was doing it to learn Latin, but you could do it for any language.
- projectorlochsa 9y agoThere's a very good book for Latin that uses that trick. Goes from zero to extremely complex Latin. Whole book is in Latin, no translations. https://www.amazon.com/Lingua-Latina-Illustrata-Pars-Familia/dp/1585104205 https://www.amazon.com/Lingua-Latina-Illustrata-Pars-Familia... The only requirement is knowledge of orthographic alphabet and how each sound is produced. Latin, fortunately, has very simple sounds compared to English or Swedish. It took me about 2 years to go through both parts and I was amazed at how easy the journey was. Could speak and write Latin fluently without issues.
- camillomiller 9y agoIs any book like that available for contemporary languages?
- wofo 9y agoI learned Dutch using a similar approach (https://www.amazon.com/Delftse-Methode-Nederlands-voor-buitenlanders/dp/9085067200/ https://www.amazon.com/Delftse-Methode-Nederlands-voor-buite...)
- mci 9y agoCheck out https://vivariumnovum.it/risorse-didattiche/propria-formazione/metodo-diretto-applicato-alle-lingue-moderne https://vivariumnovum.it/risorse-didattiche/propria-formazio... :)
- dpfu 9y agoThis is a awesome resource, thanks a lot! Some of these books are of course quite old. For example I would not recommend trying to learning contemporary German using this [1] book. Reading 'fraktur' alone can be rather challenging … [1] [Worman, J. H.: Erstes Deutsches Buch, nach der Natürlichen Methode, für Schule und Haus, American Book Company, New York 1880](https://vivariumnovum.it/edizioni/libri/dominio-pubblico/Worman%20-%20Erstes%20Deutsches%20book.pdf https://vivariumnovum.it/edizioni/libri/dominio-pubblico/Wor...)
- lkbm 9y agoThis is pretty cool. The second word I clicked was "cant"....and about half I saw were typos of "can't", so, there's some bad data in there if you're trying to learn standard english, but it's good data if you want to understand things people actually write. Anyway, time to go through and add some apostrophes to a few articles. :-)
- goshx 9y agoVery nice! Thanks for sharing! Some words are not found: https://buildmyvocab.in/affinity https://buildmyvocab.in/affinity (Just a little correction there: does not exist*)
- abhas9 9y agoThanks for pointing this out. Will fix this soon.
- AdmiralAsshat 9y agoVery cool! Although I feel that sometimes you really need a human touch to make it truly comprehended. For instance, I random clicked on "antediluvian": https://buildmyvocab.in/antediluvian/ https://buildmyvocab.in/antediluvian/ Everything here will get you a "good enough" understanding of what the word means, but this is the only one that really comes close to explaining the word's literal meaning, and it's too vague to be of much use: any of the early patriarchs who lived prior to the Noachian deluge A non-native speaker isn't going to have any idea what "Noachian" means (a native speaker probably isn't either unless they can explicitly identify "Noah" as the root), and "deluge" is part of the root of the word we're defining, so simply using the word "deluge" without explaining what it means doesn't really help. In short, this is a good groundwork, but I think it needs a human editor to push the individual definitions from "acceptable" to "correct".
- abhas9 9y agoThanks for your awesome feedback. And yes this is just the initial ground work and part of a larger experiment. We are also trying to teach English using Bollywood movies and GIFs[1]. I agree that human editing is very important and as such upvote/downvote button feature is in pipeline next. [1] https://buildmyvocab.com/ddlj.html https://buildmyvocab.com/ddlj.html
- sridca 9y agoHey cool project. Can this be used to learn French as a English speaker?
- deleted 9y ago[deleted]
- kafkaesq 9y agoIn short, this is a good groundwork, but I think it needs a human editor to push the individual definitions from "acceptable" to "correct". Good point. A StackOverflow-style upvoting system may provide exactly that.
- saurabh1728 9y agohttps://play.google.com/store/apps/details?id=com.buildmyvocab.app https://play.google.com/store/apps/details?id=com.buildmyvoc... is this your app abhas ? Quite interesting
- abhas9 9y agoYes, this app is also a part of an initiative to make learning simpler and fun.
- baby 9y ago1. I don't really understand what this is about. Having a description on the landing page would help. > Barron's 800 Words list with example sentences who is this Barron? 2. Please can you add pronunciation :D 3. words need a definition as well, not sure what some of these means even with the examples.
- sremani 9y ago> who is this Barron? http://barronstestprep.com/ http://barronstestprep.com/
- abhas9 9y agoThanks for the feedback. I understand that the current UX is very unintuitive as this was a one day hack which I published online. I will add a description and search feature soon. Regarding Pronunciation and definitions: I am exploring the possibility of integrating "Wordnet" by Princeton University. I will try to integrate it as soon as possible.
- dominotw 9y ago>who is this Barron? I think if you are asking that question you are probably not the target audience for this website.
- peterburkimsher 9y agoWhat kind of pronunciation? IPA? Phonics?
- miles 9y agoThis list is to help non-native English learners? Many native English speakers might have trouble with a few of these: abeyance, abscission, accretion, amalgamate, anodyne, antediluvian, apposite, arabesque, atavism, and avuncular.
- camillomiller 9y agoAs an Italian native with a classical studies background, this kind of words are easy for me. They're almost all Latin-derived and they usually sound very similar to the Italian equivalent. You wanna know what's hard for us? The street talk. You know like when you shoot the breeze before you really spill the beans about your shenanigans while riding shotgun on a friend's old jalopy.
- rabboRubble 9y agoThose are fairly rare words in day to day English. Some of those words I have a "feeling" for what they mean, I've definitely seen some of them in print. Most of those words I would look up if reading on a Kindle, just to check my feeling about the word. Have I used any of those words in my own writing or speech? Nope! From the linked website, the best words to learn for a learner are from the 1000 Most Common English Words section.
- jaclaz 9y agoYes, beside those you listed (that are not common in a normal conversation) there is a large number of Latin derived words in English, as a rule of thumb, there are almost always two words in English a non-latin derived one and a latin derived one having more or less the same meaning (or a near enough one). Usually native English speaking people will use the non-latin one, but of course they also generally know it (or at least have heard or read the "other" one) so they can understand you alright, but I am told that if/once you are proficient enough in English, when you - mistakenly - use the latin based ones you sound like being snob or "upper class" or very formal (and wanting to seem so). A few examples (of common words): apartment=flat < I never managed to get this right obscurity=darkness arms=weapons annually=yearly legal=lawful constructor=builder transmit/transmitted=send/sent custodian=keeper
- lacampbell 9y agoWhile I think it needs a little polishing (a lot of wikipedia sentences are fairly hairy), I really like the core idea here. Keep up the good work.
- WheelsAtLarge 9y agoThis is good stuff. I like the sentences part but I would put the definition before the sentences. This would be a great foreign language tool too.
- racl101 9y agoWhat?! No "cromulent"?
- sengork 9y agoI've always thought that the Simple English article versions of Wikipedia were always useful for non-native English speakers. https://simple.wikipedia.org/ https://simple.wikipedia.org/ Most people seem to be unaware of this Wikipedia aspect.
- sengork 9y agoFurthermore Simple English tends to be the very high level TL;DR version of associated article.
- satysin 9y agoVery cool. As others have said I think you should add definitions for the words (even a link off to an external one is fine) and pronunciation (with audio, perhaps link to forvo.com?) would be superb.
- malandrew 9y agoWhat about other languages?
- peterburkimsher 9y agoI'm working on Chinese, and everything I do gets put into http://pingtype.github.io http://pingtype.github.io This week I plan to finish making clips for words in movie subtitles.
- masteryupa_ 9y agoCould something similar be done with other languages as well, say Simplified Chinese?
- yorwba 9y agoSimplified Chinese is a bit tricky, because the Chinese Wikipedia is mostly in Traditional Chinese, because it's blocked by the Great Firewall. Otherwise, if you can find a large corpus, segment it into words and do some basic statistics, you could build something like this for any language. The most similar implementation I am aware of (using the word 中文 (Chinese) as an example) is http://ce.linedict.com/#/cnen/example?query=%E4%B8%AD%E6%96%87 http://ce.linedict.com/#/cnen/example?query=%E4%B8%AD%E6%96%...
- Baeocystin 9y agoNeat. ::clicks on a random word:: "We couldn't find any sentences for the word centripetal." So... Why is it one of the chosen few?
- opaqe 9y agoMining canonical papers/text to generate standardized tests (SAT/GRE) might be a further step. My guess is that both tests and commercial prep-material are produced by committee.
- mbrookes 9y agoCompletely OT, but is there a mathematical explanation for why, when scrolled the spaces between the words appear to form connect channels?
- Jun8 9y agoI think you're referring to word rivers (https://en.wikipedia.org/wiki/River_(typography) https://en.wikipedia.org/wiki/River_(typography)).
- peterburkimsher 9y agoIs there a word list for TOEFL and/or IELTS? I'm using a similar strategy (movies, music, Bible, articles) for studying Chinese. I'm using the TOCFL and HSK word lists. My friend uses a book with a list of 15000 vocabulary words by Morris Hill. I can't find a txt version though.
- jlebrech 9y agocan you make a container that lets you crawl any other language?
- ilamparithi 9y agoI created something similar using the Wordnik api. https://www.greedge.com/grewordlist/ https://www.greedge.com/grewordlist/
- ge96 9y agomined with a mithril axe
- ekingr 9y agoIn the same vein, for French translation, Linguee[1] uses many sources from websites of organisations that display official content in several languages (eg. the websites of the EU, of the Canadian Parliament...). The fact that it's official texts (eg. laws) makes it quite reliable. [1] http://www.linguee.fr http://www.linguee.fr
- dcsan 9y agoI find there's a lot of material for studying isolated words, but as an engineer, analyzing the sentence patterns and grammar is more interesting. I'm working on a project to do this for a database of Chinese grammar patterns. When there's enough sentence examples for each pattern as structured data, we can then make games and other learning tools. For example: yīnwèi / 因为 / because http://cgram.rikai-bots.com/grammar/yinwei http://cgram.rikai-bots.com/grammar/yinwei Now there's a magnets game to try to use that pattern: http://cgram.rikai-bots.com/magnets/?cnames=yinwei http://cgram.rikai-bots.com/magnets/?cnames=yinwei I would be happy to share the repo with anyone who's interested, or using the data to make some other language learning games. PS I did a similar thing for japanese before: JGram.org and it really helped me learn japanese quickly.
- mrstatus 9y agoi am one of learner's of English, but i can't.. tips me to get my English perfect.. http://www.mrstatus.in/himbhoomi-jamabandi-copy/ http://www.mrstatus.in/himbhoomi-jamabandi-copy/
- dadvocate 9y agoWould be more intuitive if the meaning is presented first and then example sentences