4 ms·
The paper is not published yet, as this work was done for my dissertation, however we should publish the paper in coming few weeks. As for the results they are
by Pattio 7y ago
The paper is not published yet, as this work was done for my dissertation, however we should publish the paper in coming few weeks. As for the results they are not as good as state-of-the-art methods, but they seem to be pretty competitive when compared to other open source libraries. The current problem is that complex nodes like add and skip nodes are not implemented yet, so it can only generated sequential structures. Once the paper is published I will update GitHub readme file.
- p1esk 7y agoWhat about compute resources needed?
- Pattio 7y agoRuntime on CIFAR-10 dataset for different ant counts: https://edvinasbyla.com/assets/images/deepswarm-runtime.pdf https://edvinasbyla.com/assets/images/deepswarm-runtime.pdf) Runtime compared to genetic architecture search (using similar settings): https://edvinasbyla.com/assets/images/devol-deepswarm-runtime.pdf https://edvinasbyla.com/assets/images/devol-deepswarm-runtim... The error rate on CIFAR-10, before the final training (meaning that topologies weren't fully trained and no augmentation was used): https://edvinasbyla.com/assets/images/ant-before-train.pdf https://edvinasbyla.com/assets/images/ant-before-train.pdf The error rate on CIFAR-10, before the final training compared to genetic architecture search (using similar settings): https://edvinasbyla.com/assets/images/devol-deepswarm-cifar.pdf https://edvinasbyla.com/assets/images/devol-deepswarm-cifar.... The 2 main factors that contribute to faster search are (1) ants search for architectures progressively (meaning that early architectures can be evaluated really fast), (2) ants can reuse the weights as they are associated with the graph. All test were done using Google Colab. Even though results might not seem that impressive, I am still really excited to see what will happen when ants will be allowed to search for more complex architectures which use multi-branching.
- p1esk 7y agoI just looked up the ant colony algorithm, and intuitively the idea of pheromones and path reinforcement makes a lot of sense. Are you the first one to try it for NN search?
- Pattio 7y agoTo best of my knowledge there are no published papers that use ACO for CNNs neural architecture search. However, I found three published papers that used ACO for different NAS, but they used static graphs and no heuristics.