7 ms·
Markov Chain Text Generation
- parabiii 8y agoYou can use Google search results as a Markov chain (with the ranks translated to probabilities). Then your chain is fitted to all of the visible web. Another cool use for ML is for categorical and textual data. A fitted Markov Chain can give the probability of a string occuring. Strings that are more random (such as spammy text) get a low probability. Strings that are similar (such as certain user agents) get a similar probability, without having to directly compare these.
- monokai_nl 8y agoYears ago I used Twitter streams as input for markov chain text generation. It’s interesting to recognize your own writing style in what is essentially gibberish: http://yes.thatcan.be/my/next/tweet http://yes.thatcan.be/my/next/tweet
- globuous 8y agoreally good work !! I had a great laugh reading generated text from Trump and Obama ^^
- wiggler00m 8y agoAsk HN: Markov Chain book recommendations?
- tastroder 8y agoBayesian reasoning and machine learning by Barber (2012) had nice introductory texts on most of the mechanisms at work iirc if you're looking for a lecture oriented resource.
- nafizh 8y agoConsidering RNNs have completely taken over text generation with language models, are there any papers that comprehensively compare the power of RNN and Markov models?
- xapata 8y agoIn a sense, they're equivalent. RNN's create models of probabilistic state transitions. In other words, they create Markov chain models. You might argue that a Markov chain doesn't have memory, but it's easy to incorporate "memory" by expanding the definition of a "state" to include recent history or any function of the past.
- raverbashing 8y agoNo, they're not. And if it was "that easy" to incorporate memory RNNs would not have replaced Markov Chains As an example try to build a character level Markov chain for text generation and see how that goes.
- aesthesia 8y agoThe point is that an RNN or other language model describes a stochastic process which can be seen as a Markov process with state space given by the internal state of the RNN. This doesn't say anything about the ease of implementing or learning such a model. Just that from a purely mathematical perspective, they are equivalent in power.
- raverbashing 8y agoAh I see, I agree with their equivalence in the mathematical sense.
- srean 8y agoAm curious about what other sense of 'equivalence' you had in mind when you stated with great authority that RNNs are not Markovian.
- raverbashing 8y agoIn the practical sense, implementing a RNN is easier than a Markov Chain (libraries, etc) Also RNNs can "evolve" (or be adapted) more easily to other architectures like LSTMs.
- mnagel 8y agoI did something simliar at university and (amongst other things) ran it over (simple) music scores in Lilypond format. The resulting texts could be converted to MIDI format and played by a synthesizer. It was quite interesting to listen to those. The code is still online at https://github.com/mnagel/markov https://github.com/mnagel/markov
- herval 8y agoI have a Twitter bot that reads text from random Project Gutenberg books and spits out book passages: https://twitter.com/markovian_lit https://twitter.com/markovian_lit In general, it’s a bit all over the place and convoluted, but sometimes it generates true gems. People find it via hashtags, and some folks even interact before realizing it’s a silly bot.
- menixator 8y agoTheres /r/SubredditSimulator/ on Reddit which uses markov chains. Never fails to entertain me.