3 ms·
A good way I found to store and use markov chain data was through the filesystem. Each words gets its own file, and every word that comes after that word is put
by jimmaswell 12y ago
A good way I found to store and use markov chain data was through the filesystem. Each words gets its own file, and every word that comes after that word is put in that file. The problem of the probability of one word coming after another solves itself if you just let the word be added to that list multiple times, one for each occurrence.
- wlievens 12y agoBut a file system forms a tree whereas a Markov chain is a directed graph, cycles allowed. Or are you thinking of a file system with cyclical symlinks?
- e12e 12y agoAs I understand it you would have, as an example (example as a cycle): file: As contains: I an a file: an contains: example file: example contains: example as Here "as" points to "example", and "example" points to "example" and to "as". You have your cycle: pick a random file (get example) pick a random word (get as), lookup as (open the "as"-file), pick a random word (get "as" again, then "an" then "example" … ). No links needed, the words are the links -- for US English, and words no longer that 8 characters, you could this on FAT16. (The files are vertices, and they contain a list of edges to other vertices, in the form of the name of the vertices).
- jimmaswell 12y agoYeah, that's how I had it. Newlines separating the words, and the chars that can't go in the filesystem get escaped to base64 or something.
- adamcanady 12y agoIs it then possible to model 2nd order Markov chains with every two words containing their own file? Filename: this_example Contains: is, can, etc...
- jimmaswell 12y agoThe way you described it sounds possible, yes