3 ms·
The underlying autodiff engine in smallpebble runs computations immediately, as they are defined, for the given inputs. But when modelling, you want to define a
by montebicyclelo 3y ago
The underlying autodiff engine in smallpebble runs computations immediately, as they are defined, for the given inputs. But when modelling, you want to define a model, and then use it again and again, on different inputs. The laziness is a way to do that; you can define a graph of autodiff operations (using "sp.Lazy", and "sp.Placeholder"), without running it, and then pass in values and run the graph on those operations. We end up with something similar to how models are defined in tensorflow [1].
[1] https://www.tensorflow.org/js/guide/models_and_layers#the_functional_model https://www.tensorflow.org/js/guide/models_and_layers#the_fu...
- tekknolagi 3y agoAha! Okay, thank you. And how do you pass in the inputs differently?
- montebicyclelo 3y agoIn smallpebble it's currently done by setting the value of the Placeholders, e.g. # define some graph x = sp.Placeholder() y = sp.Lazy(sp.square)(x) # and run the graph later on x.assign_value(sp.Variable(np.array([1, 2, 3]))) y_val = y.run() # the result, y_val is an sp.Variable gradients = sp.get_gradients(y_val) # run same graph with different values x.assign_value(sp.Variable(np.array([5, 6, 7]))) y_val = y.run() gradients = sp.get_gradients(y_val) (There's some more examples of this being used in training loops in the readme.)
- tekknolagi 3y agoOh, oh, okay, these placeholders are like my `input` Values. Neat! Thank you.