4 ms·
The article doesn't explain the problem of converting recursion to iteration fully. There is a lot more detail. Most recursion can be implemented using a loop
by crimsonalucard 8y ago
The article doesn't explain the problem of converting recursion to iteration fully. There is a lot more detail.
Most recursion can be implemented using a loop and a stack. If the recursive expression is self contained in the return call, then you don't need a stack.
For example:
def somefunc(x):
if x <= 0:
return 0
return somefunc(x-1)
is equal to:
def somefunc(x)
while x > 0:
x = x - 1
return x
But if the recursion isn't self contained:
def somefunc(x):
if x == 0:
return 0
return somefunc(x-1) + 1
then the iterative version
looks like this:
def somefunc(x)
stack = []
while x > 0:
stack.append(x)
x = x - 1
while len(stack) != 0:
x = x + stack[-1]
stack = stack[:-1]
return x
During Recursion your program will use an internal stack to keep track of previous context. This is called the call stack. Whether or not you use iteration or recursion for the last example you will need a stack.
The last example is actually less efficient than the recursive version because you are moving the call stack to the heap. The heap is slower than the call stack but the heap essentially has no limits so you won't encounter stack overflow.
In the first example of recursion you will actually benefit from conversion to iteration because you get rid of the stack all together in the iterative example. Some languages will be able to recognize this type of recursion and do an optimization step called "tail-recursion optimization" in which the compiler actually gets rid of the need for a the call stack. Python does not do this, however.
The story doesn't end here, however. There are examples of recursion where you cannot create a straightforward iterative version.
For example:
def somefunc(x):
if x <= 0:
return 0
return somefunc(x-1) + somefunc(x-2)
The straight forward conversion using an explicit stack looks like this:
def somefunc(x)
stack = []
while x > 0:
stack.append(lambda: somefunc(x-2))
x = x - 1
while len(stack) != 0:
x + stack[:-1]()
stack = stack[:-1]
return x
You will note that the straightforward conversion doesn't get rid of recursion. The reason is because the previous context is basically another recursive call.
You will note that the recursive version of this function actually does the same multiple recursive calls twice which leads to unnecessary repeated calculations. It is possible to optimize this repetition using something called memoization. However memoization requires and does not get rid of recursion.
It is not possible to imitate this "inn-efficiency" in an iterative form. The only way to do it is to do an optimization step similar to memoization before iterating. This step, however radically changes the way you think about the problem. It literally reverses the recursion from a top down approach to a bottom up approach. Here is what the iterative form looks like:
def somefunc(x):
store = [None for _ in range(x)]
store[0] = 0
for i in range(1, len(store)):
store[i] = store[i-1] + store[i-2] if i-2 >= 0 else 0
return store[-1]
This method is called "Dynamic programming using the tabular method."
You will note that this version of the function still allocates storage to the heap with "store"
There is still a further optimization step you can do here as well. This optimization is basically noticing that you don't need to memorize the entire store in the algorithm. You only need the previous two values.
def somefunc(x):
store_i_minus_1 = 0
store_i_minus_2 = 0
for i in range(1, len(store)):
result = store_i_minus_1 + store_i_minus_2
store_i_minus_1, store_i_minus_2 = result, store_i_minus_1
return result
You will note that the the new example has no allocation to the heap and is the full efficient iterative version.
Although my final example was applied to a recursive function that called itself twice in a single expression. The final technique I showed you can be applied to all forms of recursion to yield an iterative version WITHOUT the need to create an explicit stack in the heap. It is the ultimate optimization step in a procedural language.
One thing to note is that the final optimization can only be done in a procedural language. For a Functional programming language you will need to utilize another method.
- crimsonalucard 8y agoActually the example given in the article cannot be further optimized, even when using dynamic programming. original: def cost(s): if s <= 1: return 0 if s % 2 == 0: return 1 + cost(s // 2) return min(1 + cost(s - 1), 5) tabular: def cost(s): if s <= 1: return 0 store = [None for _ in range(s+1)] store[0] = 0 store[1] = 0 for i in range(2, s+1): store[i] = 1 + store[i//2] if i%2 == 0 else min(1 + store[i-1], 5) return store[-1] Because of this step: store[i//2], the algorithm reaches back into the store dynamically meaning that there is no fixed amount of memory that the algorithm reaches back to retrieve old results. I cannot just save the old results in a fixed size list or fixed amount of variables. So this is the furthest you can go following the tabular path.
- saagarjha 8y ago> The heap is slower than the call stack Is this true in Python? I thought the call stack was on the heap as well.
- crimsonalucard 8y agowoah. You're right. Learned something new. Why ever would they do that?
- saagarjha 8y agoThis is how most interpreted languages work. The actual hardware stack (e.g. what $rsp points to) isn’t really relevant to the Python program being executed; it’s used by the native code in the interpreter itself. Pushing a Python stack frame intermingled with a hardware stack frame would be odd (and would likely be hard to do, and weird from an ABI perspective). It’s just easier to push the call stack to a dedicated section on the heap; the Python program doesn’t know the difference anyways. Plus, you can have a much larger stack by heap allocating it.
- mcguire 8y ago"...the recursive expression is self contained in the return call" is usually spelled "...is in tail position", or perhaps "...is a tail call."