The article give the bellman equation that you need to solve. V(s)=max c sc×V′(s+c) + fc×V′(s−c) + nc×V′(s) V′(s)=min c sc×V(s−c) + fc×V(s+c) + nc×V(s) You h

The article give the bellman equation that you need to solve. V(s)=max c sc×V′(s+c) + fc×V′(s−c) + nc×V′(s) V′(s)=min c sc×V(s−c) + fc×V(s+c) + nc×V(s) You have to solve for all states simultaneously. One way to solve them is by using fixed point iteration. aka successive approximation. Pick some small alpha and update the value function in a gradient ascent kind of way. Vi+1(s) = (1-alpha)Vi(s)+alpha ( max c sc×V′i(s+c)+ fc×V′i(s−c) + nc×V′i(s) ) V′i+1(s) = (1-alpha)V'i(s)+ alpha ( min c sc×Vi(s−c) + fc×Vi(s+c) + nc×Vi(s) ) https://en.wikipedia.org/wiki/Fixed-point_iteration https://en.wikipedia.org/wiki/Fixed-point_iteration