Well for starters, fine-tuning can be done in a variety of different ways. You can pretrain your model with a larger, different dataset, or you can train an aut

Well for starters, fine-tuning can be done in a variety of different ways. You can pretrain your model with a larger, different dataset, or you can train an autoencoder that learns some useful representation of that larger dataset and use the encoder as a base for fine tuning. Another approach I've seen that was really cool is Model Distillation [0], which is basically the training of a smaller NN with the inputs and outputs of a larger NN (where the output is slightly modified to increase gradients and make training faster). [0] https://arxiv.org/pdf/1503.02531v1.pdf https://arxiv.org/pdf/1503.02531v1.pdf