4 ms·
EDIT: I put this same description into the stackoverflow page with an image which might help visualize it. I think one way to think about this is from a combin
by tolmasky 14y ago
EDIT: I put this same description into the stackoverflow page with an image which might help visualize it.
I think one way to think about this is from a combinatorics viewpoint: how many possible combinations of sorted number orderings are there? If we give the combination 0,0,0,....,0 the code 0, and 0,0,0,...,1 the code 1, and 99999999, 99999999, ... 99999999 the code N, what is N? In other words, how big is the result space?
Well, one way to think about this is noticing that this is a bijection of the problem of finding the number of monotonic paths in an N x M grid, where N = 1,000,000 and M = 100,000,000. In other words, if you have a grid that is 1,000,000 wide and 100,000,000 tall, how many shortest paths from the bottom left to the top right are there? Shortest paths of course require you only ever either move right or up (if you were to move down or left you would be undoing previously accomplished progress). To see how this is a bijection of our number sorting problem, observe the following:
You can imagine any horizontal leg in our path as a number in our ordering, where the Y location of the leg represents the value (image: http://i.stack.imgur.com/aJp4b.png http://i.stack.imgur.com/aJp4b.png ). So if the path simply moves to the right all the way to the end, then jumps all the way to the top, that is equivalent to the ordering 0,0,0,...,0. if it instead begins by jumping all the way to the top and then moves to the right 1,000,000 times, that is equivalent to 99999999,99999999,..., 99999999. A path where it moves right once, then up once, then right one, then up once, etc to the very end (then necessarily jumps all the way to the top), is equivalent to 0,1,2,3,...,999999.
Luckily for us this problem has already been solved, such a grid has (N + M) Choose (M) paths:
(1,000,000 + 100,000,000) Choose (100,000,000) ~= 2.27 * 10^2436455
N thus equals 2.27 * 10^2436455, and so the code 0 represents 0,0,0,...,0 and the code 2.27 * 10^2436455 and some change represents 99999999,99999999,..., 99999999.
In order to store all the numbers from 0 to 2.27 * 10^2436455 you need lg2 (2.27 * 10^2436455) = 8.0937 * 10^6 bits.
1 megabyte = 8388608 bits > 8093700 bits
So it appears that we at least actually have enough room to store the result! Now of course the interesting bit is doing the sorting as the numbers stream in. Not sure the best approach to this is given we have 294908 bits remaining. I imagine an interesting technique would be to at each point assume that that is is the entire ordering, finding the code for that ordering, and then as you receive a new number going back and updating the previous code. Hand wave hand wave.
- deleted 14y ago[deleted]
- greendestiny 14y agoYeah I think if you number the permutations in order from smallest to largest (ie all 0 to all 99999999) then you can update as new numbers come in by keeping track of the number (N) you've already seen and assume the other (million - N) are 0. So you just have your 10^6 bit number representing the sorted list of numbers and an 27bit integer representing the number of samples seen. The update function might be a bit ugly though :) It's probably actually a question of whether the update function could be done in the remaining memory - it certainly couldn't unpack the whole representation.
- kamaal 14y agoI think the answers to most of these seemingly impossible questions depends on discovery of isomorphic problems spaces where they can be solved a lot more easily. In your case you translated the problem to a different problem space and solved it there. The other contributors in stack overflow tend to do the same eg: Using network latency, compression etc to solve these problems. These sort of solutions become very interesting when they become isomorphic to some other real world problems.
- saucerful 14y agoIn case anyone is interested in how to prove "(N + M) Choose (M) paths": Any such path ends at (N,M). Such a path can be represented as a sequence of N+M bits, 0=right, 1=up, where there are exactly M ups. So choosing a path is identical to choosing the positions of the M ups, thus there are (N+M) choose (M) such paths. EDIT: My first proof was needlessly complicated because it dealt with paths that ended at arbitrary (N,m) but really you can just let the paths go all the way up to (N,M)-- even if the max value in the list is m < M-- and just ignore the last part of the path. It's still a bijection.