Sure, we are preparing some ways to automotate datasets creation and versionning. For now, the way to feed data is through the built-in Tensorflow numpy/pandas

Sure, we are preparing some ways to automotate datasets creation and versionning. For now, the way to feed data is through the built-in Tensorflow numpy/pandas input functions, there are some examples where this pipeline configuration is used. Otherwise there are different pipeline modules that could be used to feed the data especially for training where TF records could be faster. The way to create a TF record is still manuall, here's an image data converter that was used to create the mnist dataset: https://github.com/polyaxon/polyaxon/blob/master/polyaxon/datasets/converters.py https://github.com/polyaxon/polyaxon/blob/master/polyaxon/da... More data converters will be available soon. Once you have a record, numpy array, or a pandas Dataframe, you can basically use any operation/layer on your data by providing the feature name and the list of operation to apply. In general only operation that are necessary for data augmentation should be done on the input data pipelines, otherwise everything should be done during the creation of the TF records, to minimize the computations. One last thing for reinforcement learnign, the way we feed data is through the feed_dict, because an interaction with an environement is necessary.