3 ms·
Just recently I used a pre-trained language model to generate philosophy which was similar to my favourite philosophers. I think what impressed me was that give
by paraschopra 8y ago
Just recently I used a pre-trained language model to generate philosophy which was similar to my favourite philosophers. I think what impressed me was that given my small dataset of 5k quotes, the model ended up generating some really good quotes. Plus I trained it in just a couple of hours, not knowing much about NLP before. Here’s my tutorial https://towardsdatascience.com/generating-new-ideas-for-machine-learning-projects-through-machine-learning-ce3fee50ec2 https://towardsdatascience.com/generating-new-ideas-for-mach...
- retreatguru 8y agoWow. Fantastic article. I’d love to do the same with eastern philosophy!
- pxue 8y agofortune cookie quotes for a fun example? :)
- srean 8y agoCurious if have you tried more traditional generative models on these data sets. It seems a even a higher order Markov or a higher order hidden Markov would perform comparably. Stochastic grammars that allow recursion would be even better. With a small corpus these would run into similar difficulties but would likely perform much better than RNN/CNNs trained on the same corpus. Regardless of whether you use NN or something else your approch is the right one: start with model trained on a larger but similar corpus and then use the smaller one to modify the transition probabilities. Leaving out the "Deep" stuff will shed popular clicks on your post, probably a lot, but would get the job done with less cumulative effort. Important if for some reason you cannot use third party models. The stochastic grammars I mentioned are a lot less fiddly to train than * to vec
- paraschopra 8y agoI agree that an n-grams approach coupled with beam search may also give similar results. I did that project for fun and learning about language modeling.
- pjc50 8y agoWell, congratulations on destroying philosophy and making a fortune in the trite self-help book business. This approach to generating philosophy is the exact opposite to the philosophical tradition - there's no effort to determine if any of this stuff is "true" in any sense, there's no traditional formal reasoning model behind it; just if it sounds good. I expect to see it weaponised on political billboards and youtube videos this year. In particular, “the limits of my language are always your limits” is a horrendously mangled Fascist version of Sapir-Whorf.