2 ms·
You can't just abstract away the distributed part of it. As far as I can see, it can't be easily distributed with existing data parallel frameworks (Spark,Flink
by jamesblonde 7y ago
You can't just abstract away the distributed part of it. As far as I can see, it can't be easily distributed with existing data parallel frameworks (Spark,Flink,Beam,Dataflow). My other issue is that you are modifying the base (Mnist) code - and you will need to rewrite it again when you want to do (distributed) training. My final issue is that hyperparameters are spread throughout the code. Yes, this is better than having an external YAML file (more notebook friendly), but it is inferior to centralizing them in a single object/dict in the code. See my comments below for details.